-- Logs begin at Fri 2022-05-27 08:35:44 UTC, end at Fri 2022-05-27 10:15:04 UTC. -- May 27 09:13:26.654746 dv-cinder-825492-18-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 09:13:26.663165 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 09:13:26 2022] *** May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: nodename: dv-cinder-825492-18-iscsi May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: machine: x86_64 May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: clock source: unix May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: pcre jit disabled May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: detected number of CPU cores: 4 May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: current working directory: / May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: detected binary path: /usr/bin/uwsgi-core May 27 09:13:26.665159 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: your processes number limit is 47680 May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: your memory page size is 4096 bytes May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: detected max file descriptor number: 2048 May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: lock engine: pthread robust mutexes May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: thunder lock: enabled May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 09:13:26.665878 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 09:13:26.709294 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: Python main interpreter initialized at 0x563488eb39c0 May 27 09:13:26.709294 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: python threads support enabled May 27 09:13:26.709294 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: your server socket listen backlog is limited to 100 connections May 27 09:13:26.709294 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: your mercy for graceful operations on workers is 90 seconds May 27 09:13:26.709663 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: mapped 403077 bytes (393 KB) for 2 cores May 27 09:13:26.709740 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: *** Operational MODE: preforking *** May 27 09:13:26.710001 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: *** uWSGI is running in multiple interpreter mode *** May 27 09:13:26.710001 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: spawned uWSGI master process (pid: 80410) May 27 09:13:26.710306 dv-cinder-825492-18-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 09:13:26.711455 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: spawned uWSGI worker 1 (pid: 80419, cores: 1) May 27 09:13:26.711455 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: spawned uWSGI worker 2 (pid: 80420, cores: 1) May 27 09:13:26.711455 dv-cinder-825492-18-iscsi devstack@keystone.service[80410]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 09:13:32.335501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 09:13:32.335501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: from cryptography.utils import int_from_bytes May 27 09:13:32.336298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 09:13:32.336298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: from cryptography.utils import int_from_bytes May 27 09:13:32.379198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 09:13:32.379198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: from cryptography.utils import int_from_bytes May 27 09:13:32.379778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 09:13:32.379778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: from cryptography.utils import int_from_bytes May 27 09:13:32.384926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 09:13:32.385122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 09:13:32.385312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] command line args: [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 09:13:32.385577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 09:13:32.385750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ================================================================================ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 09:13:32.385998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] admin_token = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.386263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_dir = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.386451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.386631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_source = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.386824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] debug = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.387026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.387386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.387562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.388345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.388593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.388820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.388994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_config_append = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.389167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.389340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_dir = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.389513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.389753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.389988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.390254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.390498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.390672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.390844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.391081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.391322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.391551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.391777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.392003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.392228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.392466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.392709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.392980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.393462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.393638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] publish_errors = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.393864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.394089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.394287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.394527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.394756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.394985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] standard_threads = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.395210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.395448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.395619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.395790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_journal = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.395960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_json = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.396128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_stderr = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.396298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_syslog = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.396520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.396768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.397006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.397257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.397492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.397736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.397968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.398291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.398531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.398774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.399016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.399261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.399490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.399731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.399966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.400206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.400447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.400679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.400927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.401175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.401414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.401645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.401892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.402148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.402421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.402667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.402895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.403121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.403347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.404139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.404367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.404611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.404837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.405063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.405304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.405545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.405777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.406020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.406272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.406509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.406742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.406970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.407196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.407424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.407652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.407885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.408128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.408359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.408591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.408819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.409049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.409278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.409520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.409748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.409979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.410224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.410460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.410691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.410922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.411148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.411393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.411625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.411858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.412085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.412339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.412570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.412798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.413029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.413262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.413525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.413759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.414001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.414260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.414506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.414737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.414967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.415208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.415442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.415673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.415911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.416143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.416374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.416604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.416833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.417061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.417316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.417550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.417786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.418019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.418276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.418513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.418742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.418971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.419222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.419451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.419685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.419915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.420140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.420377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.420607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.420851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.421083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.421332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.421586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.421820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.422048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.external = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.422297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.422589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.422817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.423044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.password = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.423269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.token = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.423501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.423734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.423976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.424201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.424441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.424681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.424904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.425236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.425375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.425618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.425857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.426087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.426356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.426586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.426843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.427090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.427173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 09:13:32.427282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 09:13:32.427368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.427517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] command line args: [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 09:13:32.427678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.427766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 09:13:32.427887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.427970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ================================================================================ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 09:13:32.428053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.428128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] admin_token = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.428287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.428372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_dir = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.428526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.428615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.428712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_source = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.428804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.428879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] debug = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.428996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.429071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.429210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.429296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.429448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.429527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.429687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.429773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.429912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.429998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.430285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_config_append = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.430520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.430730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_dir = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.430923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.431080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.431158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.431325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.431410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.431564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.431650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.431792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.431871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.431981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.432060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.432155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.432273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.432395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.432549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.432624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.432717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.432855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.432961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.433086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.433198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.433316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.433432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.433546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.433678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.433788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.433924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.434031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.434151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.434330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.434428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.434635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.434845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.434934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.435006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] publish_errors = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.435201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.435292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.435455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.435535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.435632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.435778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.435859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.436023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.436112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.436291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.436369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] standard_threads = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.436526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.436604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.436764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.436912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.436912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.437152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.437275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_journal = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.437400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.437536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_json = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.437536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_stderr = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.437778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.437901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_syslog = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.438034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 09:13:32.438277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.438983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.439995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.440912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.441838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.442962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.443957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.444903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.445969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.446921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.447946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.448877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.449929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.450001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.450168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.450792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.450939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.451951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.452885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.453977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.454818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.455960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.456886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.457982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.458941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.459890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.460976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.461991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.462974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.external = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.463892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.password = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.token = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.464974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.465926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.466557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.466713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.466810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.466944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.467970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.468866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.469915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.470991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.471981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.472901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.473049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.473128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.473208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 09:13:32.473322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.473569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.473814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.474048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.474304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.474551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.474779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.475014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.475262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.475512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.475742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.475971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.476199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.476440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.476684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.476928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.477179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.477423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.477692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.477930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.478171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.478421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.478655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.478885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.479117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.479356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.479593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.479839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.480067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.480313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.480555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.480839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.481131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.481377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.481631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.481868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.482096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.482362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.482598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.482837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.483073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.483307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.483553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.483798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.484029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.484256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.484502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.484747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.484980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.485212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.485443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.485673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.485911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.486167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.486422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.486668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.486914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.487157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.487390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.487619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.487852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.488090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.488324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.488666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.488917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.489165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.489413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.489659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.489906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.490136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.490405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.490649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.490883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.491117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.491349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.491591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.491835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.492070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.492323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.492552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.492799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.493033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.493264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.493498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.493741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.493978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.494227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.494462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.494699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.494949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.495193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.495429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.495677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.495923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.496156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.496398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.496633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.496868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.497130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.497379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.497607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.497853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.498105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.498383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.498629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.498872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.499117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.499360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.499604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.499835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.500079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.500323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.500566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.500810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.501226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.501460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.501706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.501952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.502212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.502460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.502694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.502943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.503193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.503430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.503598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.503692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.503826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.503916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.504047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.504166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.504327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.504578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.504822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.505056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.505288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.505514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.505744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.505977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.506236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.506473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.506726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.506971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.507206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.507447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.507685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.507929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.508184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.508402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.508631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.508872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.509116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.509341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.509589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.509878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.510147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.510415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.510661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.510897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.511135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.511368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.511603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.511837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.512070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.512309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.512499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.512582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.513294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.513528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.513773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.514007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.514271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.514503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.514740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.514970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 09:13:32.515140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 09:13:32.541756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.542271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.542271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.543720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.544063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.560546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.581250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.581487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.600835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.619546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.653102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.653325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.657719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.657953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.724596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.724800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.724872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.725819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.750832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.751983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.752083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.752174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.752293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.764326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.764572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.777967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.778796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 09:13:32.796084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.796307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.796467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 09:13:32.796602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 09:13:32.796745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.796880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.797012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 09:13:32.797142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 09:13:32.797269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.797417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.797545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.797750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.797906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.798035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.798162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.798351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.808815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.809092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.809355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.810158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.811206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.811490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.811756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.812036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.818331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.820227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.821849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.822126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.822427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.822703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.822979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.823236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.823615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.823788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.823917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.824857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.825219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.826060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.826359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.826632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.826903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.827168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.851394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.853063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.853335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.853601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.855402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.855680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.858618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.858894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.859168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.860998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.861276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.861553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.866409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.866686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.868876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.869159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.871122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.871797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.872079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.872864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.873132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.873754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.873754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.874246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.874652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.874933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.875211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.875482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.875750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.876012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.876276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.892780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.893048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.895424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.895697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.897701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.900313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.901332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.901618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.904030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.904320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.906751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.909306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.910332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.913051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.914068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.914388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.914681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.914956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.915230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.915497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.916855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.917153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.917526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.917814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.918100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.918412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.934435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.935238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.935503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.935757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.938018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.938335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.938601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.938856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.943751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.945448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.945719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.947276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.948968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.949052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.949242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.949326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.949535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.949807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.950082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.952922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.953195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.953471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.953746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.954019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.964166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.964440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.964729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.965475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.965475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.965770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.965770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.967817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.968099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.968443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.968670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.968933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.969210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.969790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.984356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.987640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.988567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.991105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.991891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.992562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.994171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.994470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.994746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.995242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.996848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.999089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:32.999373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:32.999660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.002668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.004266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.007118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.007732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.008020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.008302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.008574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.008743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.011999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.012281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.012564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.012869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.018375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.018748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.019207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.022088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.022393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.022775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.027154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.027424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.027697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.027967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.028236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.028541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.028823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.029083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.029350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.029611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.029874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.030127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.030428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.030687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.030823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 09:13:33.030992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.031271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.031543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.031813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.032083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.032352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.032632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.032901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.033182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.033455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.033712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.033971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.034273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 09:13:33.070456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563488eb39c0 pid: 80419 (default app) May 27 09:13:33.074098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84b04-24cc-4d8a-98b2-a61e4155c6d3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:33.074403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84b04-24cc-4d8a-98b2-a61e4155c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:33.074606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x563488eb39c0 pid: 80420 (default app) May 27 09:13:33.074713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84b04-24cc-4d8a-98b2-a61e4155c6d3 None None] PATH_INFO: `/v3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:33.078979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Fri May 27 09:13:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:42.091084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:42.091598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:42.091708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:42.096005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:13:42.101423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:42.101817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:42.101970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:42.170099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: INFO dbcounter [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] Registered counter for database keystone May 27 09:13:42.191770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG oslo_db.sqlalchemy.engines [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80419) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 09:13:42.198239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writer thread running {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 09:13:42.377275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG passlib.handlers.bcrypt [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80419) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 09:13:42.377622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG passlib.handlers.bcrypt [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80419) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 09:13:42.433984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:42.436168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:42.581364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 27 09:13:42.742181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:42.811080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: INFO dbcounter [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] Registered counter for database keystone May 27 09:13:42.831083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG oslo_db.sqlalchemy.engines [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80420) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 09:13:42.837291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writer thread running {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 09:13:42.956207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5541358-e757-498d-8be8-3c870ec9c5cf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:42.960490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:42.961146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:42.961430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:42.961704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:42.969890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:43.021403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 09:13:43.021747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f35c18-e223-4df3-b93b-d6912f89bbc8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:43.032112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Fri May 27 09:13:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 09:13:43.039592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:43.054634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821b233-7ea6-41fc-8a01-86ff4dec6c3e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:43.059117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:43.060301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:43.060583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:43.060856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] PATH_INFO: `/v3/projects/822861f091c54d5980ee4872d75a26fa` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:43.062410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] RBAC: Authorizing `identity:get_project(project_id=822861f091c54d5980ee4872d75a26fa)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:43.222535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 09:13:43.223191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a33ee-9fdb-4599-8f75-2634ebac99bf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:43.227574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Fri May 27 09:13:43 2022] GET /identity/v3/projects/822861f091c54d5980ee4872d75a26fa => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:45.404812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b12-465d-4117-b0a9-af9ff098d29b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:45.405378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b12-465d-4117-b0a9-af9ff098d29b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:45.405378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b12-465d-4117-b0a9-af9ff098d29b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:45.409504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:45.415075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:45.415506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:45.415631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:45.434955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:45.436499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:45.444324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 09:13:45.467012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd3b12-465d-4117-b0a9-af9ff098d29b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:45.478874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd3b12-465d-4117-b0a9-af9ff098d29b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:45.483569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:45.484190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:45.484468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:45.484739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:45.492479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:45.494259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898f35e-14de-4d3c-88da-193c281d9735 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:45.502863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Fri May 27 09:13:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:13:45.508560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:45.517763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b709db0-9c44-445d-a395-73c20ccf16d4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:45.522147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:45.523041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:45.523317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:45.523585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:45.524975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:45.526867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0eb22-f429-43dc-b419-05363de5bfb2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:45.529013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Fri May 27 09:13:45 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:47.805426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99905c08-b6d3-4069-9b0c-bdd596252177 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:47.805939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99905c08-b6d3-4069-9b0c-bdd596252177 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:47.806043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99905c08-b6d3-4069-9b0c-bdd596252177 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:47.807402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:13:47.812815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:47.813253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:47.813357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:47.830530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:47.831653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:47.833561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8ebcc-bc03-4ce7-8680-f72283522c19 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:47.833931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8ebcc-bc03-4ce7-8680-f72283522c19 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:47.834236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8ebcc-bc03-4ce7-8680-f72283522c19 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:47.835598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:13:47.841556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:47.841837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:47.842144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:47.881999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG passlib.handlers.bcrypt [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80420) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 09:13:47.882439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG passlib.handlers.bcrypt [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80420) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 09:13:47.914864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:47.929266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:47.930894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:47.938110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:47.963563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.032496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:13:48.045200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654c7ba-e14e-4d10-a162-29d27df11e09 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.055183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.080287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.081152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.081655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.081655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.093245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:13:48.104385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:13:48.105310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:13:48.106238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:13:48.112007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.160458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53457a-45fa-4a25-8c95-d87a81bf6e94 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.163570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.196760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.198529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.199453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.199735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.200006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.219319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f23a-6086-44b0-9562-0015642e260a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.264730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.265420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.265724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.265981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.266780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:48.267092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.268527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.280115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.281656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.281789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:48.287413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.381443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:13:48.387018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.388979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e575b-435f-44e1-9c82-3ab3339e3ff6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.390589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.424445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.425332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.425614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.425892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.427273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.428784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.430503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:48.436548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.453407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.494773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f05a-b5dd-4c02-b347-4065b6bfda0a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.530638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.531446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.531728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.532013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.551279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738466-92e7-47df-adba-3f673fc8dbd8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.551666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.553171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:13:48.577025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:13:48.578284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:13:48.579165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:13:48.579165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:13:48.579165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:13:48.579165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:13:48.579165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:13:48.584314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.593114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.593915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.594190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.594486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.595706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.597185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.598838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:48.604720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:48.655295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.702127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d0ea1-b789-44d1-87df-c103f6652235 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.706743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:48.729145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213569-f256-490c-925d-8dec68972cc8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:48.739372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.740038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.740319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.740594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.741328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:48.741634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.743079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1820f4-297f-443f-b839-72da301bee48 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.763269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:48.778434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:48.778434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:48.778434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:48.778434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:48.778761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.domain.id=default, user.id=c1f81d15d7c54a30853ee3ff06b25bdd) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:48.778761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:48.778761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156848a5-5c76-4b9b-8458-70e9bab7d4cc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:48.785747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Fri May 27 09:13:48 2022] GET /identity/v3/role_assignments?user.id=c1f81d15d7c54a30853ee3ff06b25bdd&scope.domain.id=default&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 233 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:51.284177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebbe79-6806-47f1-9f41-30d57f5ea7f0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.284728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebbe79-6806-47f1-9f41-30d57f5ea7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.284728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebbe79-6806-47f1-9f41-30d57f5ea7f0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.286446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:51.291744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035abb5-22d3-4a9b-974b-c90b797d84d5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.292636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035abb5-22d3-4a9b-974b-c90b797d84d5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.293138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035abb5-22d3-4a9b-974b-c90b797d84d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.315507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4035abb5-22d3-4a9b-974b-c90b797d84d5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:51.317088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4035abb5-22d3-4a9b-974b-c90b797d84d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:51.418452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:51.453091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebbe79-6806-47f1-9f41-30d57f5ea7f0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:51.591995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0bc95-0bac-4642-8cf1-4166d5df0457 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.592288 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0bc95-0bac-4642-8cf1-4166d5df0457 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.592566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0bc95-0bac-4642-8cf1-4166d5df0457 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.593947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 13/24] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:13:51.600151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.600456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.600707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.643558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:51.644863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:51.750705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebbe79-6806-47f1-9f41-30d57f5ea7f0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:51.792255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:51.815804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 14/25] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 09:13:51.835701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:51.835874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.836131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.836413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.857347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:51.857752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:13:51.863395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6577d467-3831-40cf-a4ff-618966862445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:13:51.867230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:13:51.868123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:13:51.868862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 12/26] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:13:51.870081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe882c-0735-4fd6-9c91-32d96e6e5fe2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:51.870081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe882c-0735-4fd6-9c91-32d96e6e5fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:51.874586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe882c-0735-4fd6-9c91-32d96e6e5fe2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:51.876329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 13/27] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:51.878092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe882c-0735-4fd6-9c91-32d96e6e5fe2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:51.975553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94b64-9ffa-42d0-b131-04cec47f0b02 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:52.039135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:52.040102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.041648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.042444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.044833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:52.047829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:52.107194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe882c-0735-4fd6-9c91-32d96e6e5fe2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:52.149610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:52.150547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.151047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.151467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.152369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:52.152935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:52.154636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a2c69-8477-48bf-a78a-f9f01f0ec6ee None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:52.169813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 14/28] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:52.171569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac403b36-6642-4722-964f-a13829259956 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.172007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac403b36-6642-4722-964f-a13829259956 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.172590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac403b36-6642-4722-964f-a13829259956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.193067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ac403b36-6642-4722-964f-a13829259956 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:52.194680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ac403b36-6642-4722-964f-a13829259956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:52.231008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:52.276921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 15/29] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:52.280828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac403b36-6642-4722-964f-a13829259956 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:52.488005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 15/30] 10.222.0.48 () {60 vars in 1131 bytes} [Fri May 27 09:13:51 2022] POST /identity/v3/roles => generated 219 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:13:52.489958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:52.538279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac403b36-6642-4722-964f-a13829259956 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:52.623136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:52.683143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:52.684479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.685066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.685555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.691525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:52.693506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:52.694339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7d26-c4b2-41fc-8ed9-1fee929113ae None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:52.696591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 16/31] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:13:52 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:52.716579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:52.786469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:52.786469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.786469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.786469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.791579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:52.795507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:52.797191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:13:52.846461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:13:52.847373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:13:52.848202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:13:52.848202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:13:52.848202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:13:52.848202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:13:52.848202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 16/32] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:13:52.854090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:52.880323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93ed1-1524-4fe0-a291-23cbb380dacf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:52.921760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:52.922995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:52.923544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:52.924209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:52.925886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:52.927798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:52.930056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 17/33] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:52.936873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:53.103686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4508756-a0cd-4141-92c5-d1ab109e1d5c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:53.114644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e69f5-c79c-4313-803f-0614d5804c6d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:53.122764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:53.162457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:53.205194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:53.206815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.207340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.207862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] PATH_INFO: `/v3/domains/default/users/c1f81d15d7c54a30853ee3ff06b25bdd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.212802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c1f81d15d7c54a30853ee3ff06b25bdd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:53.214613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f4318-79cf-4a3f-99a7-73ee4d9db8d5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:53.222659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:53.223787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.224295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.224798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.226153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:53.226763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:53.229479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dac0df-4595-4803-be6e-3fb3811ae6c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:53.270840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 17/34] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:53.270840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3781f4-d1be-419e-8e69-2f7985dbfb57 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.270840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3781f4-d1be-419e-8e69-2f7985dbfb57 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.270840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3781f4-d1be-419e-8e69-2f7985dbfb57 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.271509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 18/35] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:53.273481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3781f4-d1be-419e-8e69-2f7985dbfb57 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:53.453419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 18/36] 10.222.0.48 () {58 vars in 1361 bytes} [Fri May 27 09:13:52 2022] PUT /identity/v3/domains/default/users/c1f81d15d7c54a30853ee3ff06b25bdd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:13:53.455887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da6ef-eeb0-4c7b-bf8f-ad3dfc619bd8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.456202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da6ef-eeb0-4c7b-bf8f-ad3dfc619bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.456468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da6ef-eeb0-4c7b-bf8f-ad3dfc619bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.480404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3781f4-d1be-419e-8e69-2f7985dbfb57 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:53.486722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3b0da6ef-eeb0-4c7b-bf8f-ad3dfc619bd8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:53.488505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3b0da6ef-eeb0-4c7b-bf8f-ad3dfc619bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:53.558925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:53.559713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.559959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.560232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.561167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:53.562578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:53.605351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 19/37] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:53.608932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1a25-5950-4c13-9fb6-c62abee74a15 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.609779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1a25-5950-4c13-9fb6-c62abee74a15 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.610559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b1a25-5950-4c13-9fb6-c62abee74a15 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.619934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 20/38] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:53.627633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184031cb-e416-4b8f-b0ed-0e8b3d8bc127 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.628608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184031cb-e416-4b8f-b0ed-0e8b3d8bc127 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.629544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184031cb-e416-4b8f-b0ed-0e8b3d8bc127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.660654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-184031cb-e416-4b8f-b0ed-0e8b3d8bc127 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:53.662879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-184031cb-e416-4b8f-b0ed-0e8b3d8bc127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:53.724109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 19/39] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:13:53 2022] POST /identity/v3/projects => generated 301 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:13:53.725481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:53.774406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 21/40] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:53.784489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:53.791719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a93d7-f992-4d62-b682-5547782ee639 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.792366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a93d7-f992-4d62-b682-5547782ee639 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.792944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a93d7-f992-4d62-b682-5547782ee639 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.794826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 22/41] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:53.797524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a93d7-f992-4d62-b682-5547782ee639 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:53.832627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:53.885891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482ecbb-29a8-4c53-8b06-f3845645cc7f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:53.927222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:53.928390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.928985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.929560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.929985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a93d7-f992-4d62-b682-5547782ee639 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:53.930901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:53.933862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740ce51-0f34-48a2-af26-2718a39683ef None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:53.972928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:53.973749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:53.974046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:53.974415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:53.975144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:53.976807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:54.002367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 20/42] 10.222.0.48 () {60 vars in 1131 bytes} [Fri May 27 09:13:53 2022] POST /identity/v3/roles => generated 225 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:13:54.004345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:54.004858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:54.005335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:54.030977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:54.033102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:54.063926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 23/43] 10.222.0.48 () {60 vars in 1131 bytes} [Fri May 27 09:13:53 2022] POST /identity/v3/roles => generated 223 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:13:54.183162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 21/44] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:54.216108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:54.348646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:54.382578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5fd1a8-c867-417a-a830-b2e9f911e042 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:54.443736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:54.444569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:54.444862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:54.445146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:54.447036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:54.448774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a3302-4e66-4870-9a08-79b2de7b24a8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:54.451082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 24/45] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:54.461027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:54.493620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:54.620179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45fcb9-924a-4940-9211-606225e65d0c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:54.659486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:54.660163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:54.660450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:54.660730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:54.661468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:54.662999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c8318-d6b6-47dc-961c-d73114b250a7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:54.756020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 22/46] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:13:54 2022] POST /identity/v3/projects => generated 312 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:13:56.120831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c8e29-ac38-473a-882c-b86ec37bb494 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.121379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c8e29-ac38-473a-882c-b86ec37bb494 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.122566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c8e29-ac38-473a-882c-b86ec37bb494 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.124104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 25/47] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:56.133751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.134362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.134587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.153666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:56.154913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:56.223034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:56.227664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 23/48] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:13:56.252234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c8e29-ac38-473a-882c-b86ec37bb494 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:56.360505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c8e29-ac38-473a-882c-b86ec37bb494 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:56.396361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:56.397083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.397373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.397764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.409125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:13:56.416906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:13:56.427124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:13:56.427967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 26/49] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:13:56.428577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:56.523530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca070778-b65c-4dc9-a530-79c7ba7e33f2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:56.557204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:56.557870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.558176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.558477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.559220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:56.560093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:56.561515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:56.572311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 24/50] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:56.577757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:56.593706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:56.678472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17c8a2-277f-4c8a-8d86-00ca607e7c87 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:56.712415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:56.713264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.713544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.713815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.715031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:56.716527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:56.718166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 27/51] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:13:56 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:56.723542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:56.787495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:56.826464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a564-f4a6-4e91-ae52-ecc6299ecf24 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:56.863868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:56.864676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:56.864958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:56.865231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:56.866511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:56.867994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe796b9-5948-4afc-b79a-0f27c760af00 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:56.869623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 25/52] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:13:56.875901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:56.957318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:56.975777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669b05b-4be4-45bc-8bdd-df2fe2f22166 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:57.010341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:57.010999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:57.011286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:57.011560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:57.012395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.domain.id=default, user.id=c1f81d15d7c54a30853ee3ff06b25bdd) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:13:57.013219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:57.014697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d8103-3fb0-42a8-8f95-7a02570a3e78 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:57.022928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 28/53] 10.222.0.48 () {56 vars in 1322 bytes} [Fri May 27 09:13:56 2022] GET /identity/v3/role_assignments?user.id=c1f81d15d7c54a30853ee3ff06b25bdd&scope.domain.id=default&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:57.031767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fb16c-f514-4ebb-9ef6-a70eceb7fc5a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:57.032167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fb16c-f514-4ebb-9ef6-a70eceb7fc5a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:57.032534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fb16c-f514-4ebb-9ef6-a70eceb7fc5a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:57.034034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 26/54] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:57.040012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:57.040299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:57.040580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:57.059257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:57.060529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:57.133101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 29/55] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:57.157578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fb16c-f514-4ebb-9ef6-a70eceb7fc5a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:57.267320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fb16c-f514-4ebb-9ef6-a70eceb7fc5a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:57.296841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:57.302822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:57.303609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:57.303896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:57.304175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:57.305336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:57.306935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c41cb-5e67-4c24-83d3-6f2bc7570030 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:57.308559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 27/56] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:13:57.314911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:13:57.388990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:13:57.414161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5453713-b9bd-4652-adeb-372a496ad260 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:13:57.448844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:13:57.449492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:57.449774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:57.450105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:57.450836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:13:57.452285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a3dc9-0d1d-42e9-85df-750830f8e865 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:13:57.537032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 30/57] 10.222.0.48 () {60 vars in 1140 bytes} [Fri May 27 09:13:57 2022] POST /identity/v3/projects => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:13:59.810434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc0fb-b30c-4b63-9f05-058cfa2584e0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:59.810942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc0fb-b30c-4b63-9f05-058cfa2584e0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:59.811025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc0fb-b30c-4b63-9f05-058cfa2584e0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:59.812333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 28/58] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:13:59.817415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:13:59.817798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:13:59.817968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:13:59.836007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:13:59.837101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:13:59.900459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 31/59] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:13:59.924114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bc0fb-b30c-4b63-9f05-058cfa2584e0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:00.030620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bc0fb-b30c-4b63-9f05-058cfa2584e0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:00.064258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:00.065042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:00.065328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:00.065603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:00.066797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:00.068255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761365ce-a667-487a-8cdc-0bd41480137f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:00.069870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 29/60] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:00.075643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:00.130086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:00.172243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df58753-ca96-47f4-a1ce-0c6523cbfd25 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:00.205767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:00.206451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:00.206739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:00.207020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:00.207739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:00.209196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b36f6c-583c-4cfd-980d-b79316c804ff None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:00.323614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 32/61] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:00 2022] POST /identity/v3/users => generated 275 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:03.910754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759b6d-3246-4734-9dcd-54aed9039534 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:03.911844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759b6d-3246-4734-9dcd-54aed9039534 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:03.912346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba759b6d-3246-4734-9dcd-54aed9039534 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:03.913994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 30/62] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:03.920458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c81f91-90b4-4e66-90de-30a4876062da None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:03.921630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c81f91-90b4-4e66-90de-30a4876062da None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:03.922116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c81f91-90b4-4e66-90de-30a4876062da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:03.954078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-38c81f91-90b4-4e66-90de-30a4876062da None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:03.955653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-38c81f91-90b4-4e66-90de-30a4876062da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:04.066395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 33/63] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:04.085198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d573-56e3-4c29-bb59-25dc066a9d3b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.085403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a7dc4-6775-439a-80a6-24cdabbdc976 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.085612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a7dc4-6775-439a-80a6-24cdabbdc976 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.085860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a7dc4-6775-439a-80a6-24cdabbdc976 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.086355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d573-56e3-4c29-bb59-25dc066a9d3b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.086851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15d573-56e3-4c29-bb59-25dc066a9d3b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.087486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 34/64] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:04.089247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 31/65] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:04.095292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15d573-56e3-4c29-bb59-25dc066a9d3b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.095787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0527e-4bcd-46be-858a-b9c2941dad47 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.096105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0527e-4bcd-46be-858a-b9c2941dad47 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.096410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0527e-4bcd-46be-858a-b9c2941dad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.122310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5fb0527e-4bcd-46be-858a-b9c2941dad47 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:04.123554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5fb0527e-4bcd-46be-858a-b9c2941dad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:04.169671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.201133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.206712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 35/66] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:04.210459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321274f-6710-4fc1-bf51-5bf48684772a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.210459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321274f-6710-4fc1-bf51-5bf48684772a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.210459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321274f-6710-4fc1-bf51-5bf48684772a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.229987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7321274f-6710-4fc1-bf51-5bf48684772a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:04.232136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7321274f-6710-4fc1-bf51-5bf48684772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:04.264429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15d573-56e3-4c29-bb59-25dc066a9d3b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.303622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.304450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.304725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 36/67] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:04.306113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab2ee3-6ce8-4bcb-b7d8-3b22f918f75a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.306440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab2ee3-6ce8-4bcb-b7d8-3b22f918f75a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.306724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab2ee3-6ce8-4bcb-b7d8-3b22f918f75a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.308306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 37/68] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:04.311114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab2ee3-6ce8-4bcb-b7d8-3b22f918f75a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.314589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.314975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.329511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-aa4cc868-cfec-487b-851c-c0b48121d9ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:04.337928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:04.339032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:04.339820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 32/69] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:04.341056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.341319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.341597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.358994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:04.360233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:04.418266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab2ee3-6ce8-4bcb-b7d8-3b22f918f75a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.420478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.428644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 33/70] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:04.429513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.457123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.458024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.458408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.458690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.470580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e40a6c9d-a886-4905-a72f-e44b694a76bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:04.478505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:04.479379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:04.480173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 38/71] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:04.481195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa88dc-6ffc-4d91-9755-4004afae640b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.481449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa88dc-6ffc-4d91-9755-4004afae640b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.481742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa88dc-6ffc-4d91-9755-4004afae640b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.486310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 39/72] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:04.486310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa88dc-6ffc-4d91-9755-4004afae640b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.490417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.538027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9d081-7036-4955-8fd0-00e64fea2b07 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.575889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.576689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.576982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.577325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.588398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:04.593865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa88dc-6ffc-4d91-9755-4004afae640b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:04.596268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:04.597233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:04.597989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 34/73] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:04.598667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.631862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.632527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.632809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.633085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.633838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:04.634146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:04.635579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:04.643953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 40/74] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:04.644784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.670829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.702119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853561b8-ed98-4266-a5c1-ab8305baa069 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.718036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.736097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.736885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.737164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.737435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.738676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:04.740145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da0584-eee7-41dd-9f73-ebc4fc8d17e1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:04.741766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 35/75] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:04.742977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.743330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.743609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.752032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4e6b-52ca-426f-9edc-55dbef510136 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.760997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:04.762227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:04.786828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.787486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.787773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.788237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.788985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:04.789286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:04.790718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:04.799130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 41/76] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:04.799945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.830450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 36/77] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:04.846424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.892230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.906332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a08d90-f34e-4426-92e2-68d6e0612269 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.936337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885dd0a-a512-43f2-9701-4db4a380473b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:04.940367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.941035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.941315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.941590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.942394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:04.942709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:04.944119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:04.952177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:04.953698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 42/78] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:04.954538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:04.975678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:04.976548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:04.976821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:04.977098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:04.991225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:04.992689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:04.994410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 37/79] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:04.995176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.057518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675c3d6-6226-40fc-8174-245604b46254 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.092405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.093069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.093352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.093629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.094290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd1e30-bb20-4f93-86e6-c304041e298e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.094388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.095824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.128783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.129655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.129938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.130232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.131450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.133004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.134682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:04 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:05.135452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.176525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.204072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 43/81] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:04 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:05.204816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.208839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.245565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b7d-9174-4da9-b703-18e32044252d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.288838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.289677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.289961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.290267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.304250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:05.312994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:05.313931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:05.314764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 39/82] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:05.315425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.331442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341d3a5-1e63-4e7b-875f-78ba3c95bedf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.369696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.370628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.370915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.371195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.373089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.374619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.376559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 44/83] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:05.377174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.439277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.442445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345617a-4bc9-4115-b441-dd02c8cd8883 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.507695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.508095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.508376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.508803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.509998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.555541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd83bd-23d0-49e2-ad08-1926c49ecaef None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.601344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.603569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.606780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.607643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.608113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.608680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.683561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.685402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.719910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.724214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 40/84] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:05.725156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.762763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.796974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 45/85] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:05.798251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.843853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b9f15-b940-4640-b57e-dcf67e6a14d1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.884090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.884791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.885048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.885327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.886095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:05.886449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:05.888012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:05.897650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 41/86] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:05.898645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:05.909264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ba25e-9539-4ce4-8096-2a2e6c03c02e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:05.945308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:05.947035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:05.947534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:05.948218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:05.968746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:05.993553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:06.011515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e522c-1f1b-4624-b03c-a4bd3beeb85d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.021869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.023393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.050872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.052083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.052641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.053083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.054675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.056464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.058615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 42/87] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.059783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.153778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 46/88] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:06.159022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.188424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2164ee8-cd96-4eb3-b095-4d1b00e83a2b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.235507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:06.242433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:06.249166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.250803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.251299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.252059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.253824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.256454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.259015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 43/89] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.260028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.276922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae5ef3-7df5-43b2-b89f-666474ef01fc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.316501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.317635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.317926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.318384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.319716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.321704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.323867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 47/90] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.324639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.397058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a90637-72a5-4fa2-8ff3-cb623ddca144 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.435389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.436309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.436436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.436814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.437718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:06.438102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.439808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.446344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 44/91] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.447305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.469890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe41a6-dc53-4179-9f76-99661bcb2d48 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.522375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:06.535795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.536783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.537288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.537761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.540940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.542770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.551627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 48/92] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.552990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.588644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:06.630876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec04a7-4fa3-44a5-aa67-d14934f96752 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.692489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.693186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.693468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.693740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.694631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=c1f81d15d7c54a30853ee3ff06b25bdd) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:06.694949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:06.696783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:06.709357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 45/93] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/role_assignments?user.id=c1f81d15d7c54a30853ee3ff06b25bdd&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:06.710253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:06.805985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c110064-1ec4-4db8-bc80-0cfc072b81f0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:06.895138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:06.896224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:06.896727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:06.897211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:06.974278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbab80-3194-4981-aa74-71912deb4e87 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:07.035947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:07.056322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:07.057353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:07.057858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:07.058388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:07.059578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:07.060426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:07.061520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:07.063616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:07.065310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:07.081623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 46/94] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:07.094251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:07.214103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 49/95] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:07.220909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:07.385429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7e5a1-e5e1-40b7-8794-a6f806b813e5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:07.432226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:07.433176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:07.433455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:07.433773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] PATH_INFO: `/v3/projects/0184a3906c2b47fab49e4aa9a77386ce` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:07.439244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] RBAC: Authorizing `identity:get_project(project_id=0184a3906c2b47fab49e4aa9a77386ce)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:07.440753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd800113-a670-4e7d-b525-052ae71cd240 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:07.442897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 47/96] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:07 2022] GET /identity/v3/projects/0184a3906c2b47fab49e4aa9a77386ce => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:07.449157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:07.626446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889ac99-f5c9-4816-8f65-768bd868c54b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:07.637485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:07.673977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:07.674684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:07.674971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:07.675241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:07.676091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0184a3906c2b47fab49e4aa9a77386ce, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:07.676406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:07.677882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6911971-bc59-47e7-b6b8-910bda5b00aa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:07.685231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 50/97] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:07 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=0184a3906c2b47fab49e4aa9a77386ce&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:09.019224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69e86-661d-4b1a-a0ee-cc4059e5e2bc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.020256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69e86-661d-4b1a-a0ee-cc4059e5e2bc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.020758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69e86-661d-4b1a-a0ee-cc4059e5e2bc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.022717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 48/98] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:09.029503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea0e2-7f5a-4b28-89e1-aa7816435c4d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.030457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea0e2-7f5a-4b28-89e1-aa7816435c4d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.030926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ea0e2-7f5a-4b28-89e1-aa7816435c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.055936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e77ea0e2-7f5a-4b28-89e1-aa7816435c4d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:09.057930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e77ea0e2-7f5a-4b28-89e1-aa7816435c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:09.161645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 51/99] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:09.203902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69e86-661d-4b1a-a0ee-cc4059e5e2bc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:09.344105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:09.431086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69e86-661d-4b1a-a0ee-cc4059e5e2bc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:09.460746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb960-b291-41bb-b3fe-98f57bbf33eb None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.461280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb960-b291-41bb-b3fe-98f57bbf33eb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.461706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb960-b291-41bb-b3fe-98f57bbf33eb None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.463710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 52/100] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:09.475779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.476350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.476784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.503063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:09.503063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.503063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.503063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.515441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:09.517009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:09.531503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:09.551199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:09.552113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:09.552881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:09.555036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 49/101] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:09.571172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:09.664562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 53/102] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:09.744537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:09.794981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:09.838479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e362c-d3fd-43bb-91cf-c972c253b470 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:09.878655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28d4bb-62e3-4ba0-9519-382c2ae0f993 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:09.894847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:09.927499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:09.928594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.929029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.929448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.931842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:09.934186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:09.937153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 54/103] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:09.945467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:09.946649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:09.947534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:09.948413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:09.949210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:09.950216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:09.952545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:09.954225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:09.975369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 50/104] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:09.982581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:10.172668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:10.179349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08977d-5f32-4fc4-80cf-96c7fa32dd28 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:10.222436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c94bcc-9edd-4448-b2cd-6e4623ddfb41 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:10.282081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:10.283084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.283518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.283937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.284831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:10.286534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52613f-7e16-47a2-afd8-39a95713a053 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:10.316493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:10.317656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.318110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.318557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] PATH_INFO: `/v3/users/85d4d1ec048049ac9071edf9e949d9d2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.347690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] RBAC: Authorizing `identity:get_user(user_id=85d4d1ec048049ac9071edf9e949d9d2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:10.349571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:10.351931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 51/105] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:09 2022] GET /identity/v3/users/85d4d1ec048049ac9071edf9e949d9d2 => generated 289 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:10.358305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:10.391126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 55/106] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:14:09 2022] POST /identity/v3/projects => generated 302 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:10.457657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:10.540886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ad863-6527-473c-9725-540b9080e093 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:10.591939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:10.592613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.592902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.593183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.619364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7d76-39b5-460b-a671-5385ef365ad9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.619666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7d76-39b5-460b-a671-5385ef365ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.619931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7d76-39b5-460b-a671-5385ef365ad9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.621345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 56/107] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:10.642021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826630e2-987b-4cb8-a0fd-2534e28c2998 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.642378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826630e2-987b-4cb8-a0fd-2534e28c2998 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.642614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826630e2-987b-4cb8-a0fd-2534e28c2998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.678848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-826630e2-987b-4cb8-a0fd-2534e28c2998 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:10.680105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-826630e2-987b-4cb8-a0fd-2534e28c2998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:10.767632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:10.770486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:10.859964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 57/108] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:10.862237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d32b-fd5f-4f90-83eb-36bce49f8ded None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.864154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d32b-fd5f-4f90-83eb-36bce49f8ded None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.864662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197d32b-fd5f-4f90-83eb-36bce49f8ded None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.867369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 58/109] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:10.872184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:10.877641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:10.878151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:10.878745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:10.922637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 52/110] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:10.924122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:10.927226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:10.929686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:11.047367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5a6fc-48cc-4a37-a87c-7dbb008bbe95 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.080871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 59/111] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:11.082124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.088617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.088617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.088617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.088617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.100598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:11.108129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:11.110725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:11.111632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:11.112381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 53/112] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:11.119805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.136246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:11.267562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c7fe8-af7e-45ae-9551-0e7f4206ad5b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.316263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.318080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.318678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.319099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.320324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:11.320786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:11.323574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:11.327092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370bdd6-3f69-4c66-adde-7347af8471e5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.337058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 54/113] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:11.342256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.398235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.399982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.400788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.401111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.403468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:11.406540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86e624-9854-41fc-bafc-bed671f4fb77 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:11.419129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 60/114] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:11.420948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:11.423690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0910-f68e-4c25-87ac-8852fd4896af None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.424203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0910-f68e-4c25-87ac-8852fd4896af None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.424713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a0910-f68e-4c25-87ac-8852fd4896af None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.428557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 61/115] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:11.429058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a0910-f68e-4c25-87ac-8852fd4896af None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.463745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c90f4-6cf0-4921-92f4-480303e1f218 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.505582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.506781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.507267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.507738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.520034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:11.530293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:11.531204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:11.531962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:11.534471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 55/116] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:11.535527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.604553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:11.644645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a0910-f68e-4c25-87ac-8852fd4896af None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.657634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bda3b-1e8f-40d2-8b22-dda266236bee None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:11.682218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:11.698664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.700040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.700530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.701565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.702684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=85d4d1ec048049ac9071edf9e949d9d2) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:11.703468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:11.705489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25fe51-02b1-4b1d-97b0-e7af8406800d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:11.713492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 56/117] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/role_assignments?user.id=85d4d1ec048049ac9071edf9e949d9d2&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:11.715273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.716307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.716782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.721814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:11.724790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:11.724790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:11.725150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:11.733982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:11.736851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:11.739271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:11.745006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:11.747300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 62/118] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:11.749132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:11.909651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 57/119] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:11.918861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.049220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2cff6-23d8-4791-bff4-fcd1e1430fe4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.162156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef0c71-19e5-4269-b6d6-739dcbc43c97 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.164354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.165425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.165878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.166357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.167653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:12.168162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:12.170616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:12.204719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:12.207665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 63/120] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:12.209399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.238450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:12.271548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.272726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.272962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.273343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.279606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:12.281545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:12.283249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 58/121] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:11 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:12.284149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.456046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060d389-7f25-461e-b939-009bd65932b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.462531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a79fcd-2b36-4295-964e-c3decf91e868 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.496107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.497531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.497854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.498128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.499457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:12.500969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:12.503051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 59/122] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:12 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:12.508051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.526144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.527607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.528714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.529208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.552769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3b4d6d2b-2047-403a-aa43-de01f8bd7b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:12.563508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:12.564541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:12.565335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 64/123] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:12.566295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3fde4-56cf-4761-974a-7294e695ce44 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.566447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3fde4-56cf-4761-974a-7294e695ce44 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.566727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3fde4-56cf-4761-974a-7294e695ce44 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.574647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 65/124] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:12.574647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3fde4-56cf-4761-974a-7294e695ce44 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.591307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:12.642722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e442c-e52a-4917-a431-7ac0b491eab2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.655559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:12.688194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.689681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.690172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.690674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/users/f83fca0dd8e84a8191f04222fdc03da6/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.693049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3fde4-56cf-4761-974a-7294e695ce44 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.693836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, user_id=f83fca0dd8e84a8191f04222fdc03da6, role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:12.696107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d724b8-77b8-48a9-ab70-8eef398e68e7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:12.734010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:12.734883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.735169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.735464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.736731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:12.738283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:12.744964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 66/125] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:12 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:12.744964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:12.772504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 60/126] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:12 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/users/f83fca0dd8e84a8191f04222fdc03da6/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:12.774458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:12.775018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:12.775530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:12.798914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:12.800811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:12.967497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e49df9-a7a5-4c27-8df0-0a28cfcbcbdc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:12.978287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 61/127] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:12.984235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.049904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.050987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.051488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.051952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.052900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:13.053399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:13.055034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:13.057858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:13.062817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:13.069003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 67/128] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:13.070157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.129940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c498ee-bf63-49ec-a597-2142c26dbbd0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:13.173015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.174394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.174898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.175353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/users/c1f81d15d7c54a30853ee3ff06b25bdd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.177824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, user_id=c1f81d15d7c54a30853ee3ff06b25bdd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:13.179687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:13.212103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbea65-0b40-46ec-a10e-52ca107f2e67 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:13.251093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.251903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.252191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.252470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.265433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:13.273414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:13.274302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:13.275064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 68/129] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:13.275690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.288725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 62/130] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:12 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/users/c1f81d15d7c54a30853ee3ff06b25bdd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:13.304435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.385207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:13.455608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a90dad-8e9d-4eec-8fc1-e898c508dec6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:13.515749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.516557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.516842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.517119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.527459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:13.529063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:13.531241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 69/131] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:13 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:13.548270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.582135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:13.594569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6006a5-9a7a-4f52-b604-998df6dc7cd3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:13.715491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.716552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.716968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.717376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.718598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:13.719083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:13.721403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:13.742024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 63/132] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:13.753763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.823975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e15e5-edf4-4619-ba05-ed8ca661fc8a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:13.881764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:13.900023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:13.900903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:13.901179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:13.901461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:13.904948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:13.906505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:13.908432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 70/133] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:13 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:13.920421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:13.993801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352ed19-8235-4a99-9b3a-fdaed4426d35 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:14.114457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:14.117986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.118817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.119603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.125766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:14.129783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:14.140017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 64/134] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:13 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:14.149242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:14.232625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:14.233807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd812f-896f-42f1-a885-30058b221d60 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:14.299652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:14.300563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.300851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.301115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/users/f83fca0dd8e84a8191f04222fdc03da6/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.311489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, user_id=f83fca0dd8e84a8191f04222fdc03da6, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:14.313047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81770f4f-25a6-484e-a178-484a8de2e3d6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:14.357447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0085-fbc1-4f58-8b4e-baa5c62a0f28 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:14.396592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:14.397448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.397784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.398069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] PATH_INFO: `/v3/projects/0184a3906c2b47fab49e4aa9a77386ce` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.406880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] RBAC: Authorizing `identity:get_project(project_id=0184a3906c2b47fab49e4aa9a77386ce)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:14.409627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 71/135] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:13 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/users/f83fca0dd8e84a8191f04222fdc03da6/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:14:14.409627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8460b-096c-43ee-8134-e2d1f6b70984 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.409627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8460b-096c-43ee-8134-e2d1f6b70984 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.409627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8460b-096c-43ee-8134-e2d1f6b70984 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.410941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c93e7-bdac-4e3d-b2fc-db1fe33f3f4e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:14.411199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 72/136] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:14.414575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 65/137] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:14 2022] GET /identity/v3/projects/0184a3906c2b47fab49e4aa9a77386ce => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:14.421947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8460b-096c-43ee-8134-e2d1f6b70984 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:14.424168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.424922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.425635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.475351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:14.476649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:14.534557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:14.666414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 66/138] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:14.666991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8460b-096c-43ee-8134-e2d1f6b70984 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:14.724033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:14.724930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.725206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.725476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] PATH_INFO: `/v3/projects/0184a3906c2b47fab49e4aa9a77386ce/users/f83fca0dd8e84a8191f04222fdc03da6/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.732065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] RBAC: Authorizing `identity:create_grant(project_id=0184a3906c2b47fab49e4aa9a77386ce, user_id=f83fca0dd8e84a8191f04222fdc03da6, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:14.733661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:14.743278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:14.797243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:14.822233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 73/139] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:14 2022] PUT /identity/v3/projects/0184a3906c2b47fab49e4aa9a77386ce/users/f83fca0dd8e84a8191f04222fdc03da6/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:14.914726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e964a-1ec8-4af8-bbbf-1776423d95b3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:14.962133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:14.962919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:14.963200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:14.963476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:14.964888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:14.966672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804327d3-3814-4b5f-b157-0e01f097f366 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:14.969978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 67/140] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:14.977219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:15.163652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0e0b-c805-4734-b9ee-7e436e10977f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:15.194469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:15.222165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:15.223189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:15.223673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:15.224143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:15.225068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:15.226783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9bce2-af5a-4552-9652-563fd358c2e8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:15.337987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 74/141] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:14 2022] POST /identity/v3/users => generated 283 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:15.861820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bec92-a6e0-4e29-b858-c3f8c05c5bcc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:15.862341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bec92-a6e0-4e29-b858-c3f8c05c5bcc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:15.862341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bec92-a6e0-4e29-b858-c3f8c05c5bcc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:15.863909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 68/142] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:15.869880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:15.870851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:15.871399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:15.894810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:15.896359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:15.979407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 75/143] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:16.002065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bec92-a6e0-4e29-b858-c3f8c05c5bcc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:16.127589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bec92-a6e0-4e29-b858-c3f8c05c5bcc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:16.130994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:16.168851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:16.169676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:16.169962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:16.170279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:16.187479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:16.195719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:16.197479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:16.198858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 69/144] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:16.204154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:16.360834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc712c-d7a9-4fd0-993b-853a2a9f7b0c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:16.362958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:16.407886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:16.408606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:16.411441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:16.411784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:16.412526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:16.412835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:16.414377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:16.424567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 76/145] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:16.430690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:16.576155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a088-b152-42f6-807d-2eaae7be9705 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:16.628608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:16.629504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:16.629789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:16.630063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] PATH_INFO: `/v3/users/85d4d1ec048049ac9071edf9e949d9d2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:16.631454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] RBAC: Authorizing `identity:get_user(user_id=85d4d1ec048049ac9071edf9e949d9d2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:16.632972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651268f8-50d9-4c4d-9702-c5cfabc1a035 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:16.635099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 70/146] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:16 2022] GET /identity/v3/users/85d4d1ec048049ac9071edf9e949d9d2 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:16.642177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:16.656308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc236-b24b-470c-a7ca-0fa19d310e9e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:16.656697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc236-b24b-470c-a7ca-0fa19d310e9e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:16.656916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dc236-b24b-470c-a7ca-0fa19d310e9e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:16.658387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 71/147] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:16.677490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d29b-f290-40f7-8f3a-409d991d4d88 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:16.677905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d29b-f290-40f7-8f3a-409d991d4d88 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:16.678054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d29b-f290-40f7-8f3a-409d991d4d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:16.723066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cc77d29b-f290-40f7-8f3a-409d991d4d88 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:16.725143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cc77d29b-f290-40f7-8f3a-409d991d4d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:16.798714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:17.057935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 72/148] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:17.069612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd0922-ecba-4e04-9370-e6e0a078cf53 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:17.140765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1a48-0ce4-41d2-9fa9-c8d4dce2643c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.141653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1a48-0ce4-41d2-9fa9-c8d4dce2643c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.142440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1a48-0ce4-41d2-9fa9-c8d4dce2643c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.150232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 73/149] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:17.159959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1a48-0ce4-41d2-9fa9-c8d4dce2643c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:17.211686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:17.212587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.212867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.213136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.219214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:17.220792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55738a07-bbf2-4131-b42a-878c930fcf14 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:17.223076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 77/150] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:16 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 581 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:17.227521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.227977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.228524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.275451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:17.276666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:17.332972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:17.440125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1a48-0ce4-41d2-9fa9-c8d4dce2643c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:17.460382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 78/151] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:17.461656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:17.504307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:17.505396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.505827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.506309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.519373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:17.532835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:17.533856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:17.534703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:17.536949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 74/152] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:17.548289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:17.582821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:17.693516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227844f9-f45b-4427-9b87-4e014d13d77f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:17.733827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:17.734914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.735344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.735792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.748322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:17.757904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:17.758905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:17.759764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:17.762359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 75/153] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:17.764222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:17.858722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:17.876269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a6663-02c2-4802-8af3-65b695a3e90e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:17.887090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a4429-c55a-4a5d-9021-d00b317776ac None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:17.927414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:17.928428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.928925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.929382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.930390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:17.931041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:17.932663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:17.943209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 76/154] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:17.945042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:17.989506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:17.990751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:17.991217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:17.991671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/users/85d4d1ec048049ac9071edf9e949d9d2/roles/7e5337e63b0b4809b748baf1e1396bf6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:17.995907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, user_id=85d4d1ec048049ac9071edf9e949d9d2, role_id=7e5337e63b0b4809b748baf1e1396bf6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:17.998296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:18.080356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 79/155] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:17 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/users/85d4d1ec048049ac9071edf9e949d9d2/roles/7e5337e63b0b4809b748baf1e1396bf6 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:18.081985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:18.126681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436254f-f4f1-4637-89c0-28a8b4d06061 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:18.143942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:18.244666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:18.245270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:18.245551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:18.245823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:18.247231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:18.247576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:18.249056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:18.255827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:18.275971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 77/156] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:18.283494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:18.302908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2766b4-0357-44d1-ac2d-465fc44f756b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:18.387941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:18.388839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:18.389115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:18.389384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:18.392244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:18.393759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:18.395589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 80/157] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:18 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:18.424013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:18.618873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:18.626331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd6d2e-a5d8-49f5-a420-5f54d3a3787b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:18.642238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab07ac-276c-42be-9fe7-838cd7eeee73 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:18.684401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:18.685696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:18.686180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:18.686704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] PATH_INFO: `/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:18.688239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] RBAC: Authorizing `identity:get_user(user_id=c1f81d15d7c54a30853ee3ff06b25bdd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:18.690011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:18.691982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 78/158] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:18 2022] GET /identity/v3/users/c1f81d15d7c54a30853ee3ff06b25bdd => generated 247 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:18.701334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:18.716580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:18.717613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:18.717613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:18.717888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:18.788316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:18.833317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c5639-4401-4097-9557-7b3db39ddc4f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:18.877090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:18.878160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:18.878890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:18.879419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:18.895166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:18.896605 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:18.969336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:18.971698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:19.062804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:19.123484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:19.126163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 79/159] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:14:19.134605 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 81/160] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:14:19.141035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:19.143825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:19.297512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ea62b-d6f9-45f3-a8a5-64fa45f20711 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:19.298930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6089c-aed1-4966-860e-c798fbdeeb02 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:19.343527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:19.344557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:19.344942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:19.345320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:19.346751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:19.348496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:19.350863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 80/161] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:19 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:19.353164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375382c-58b2-448a-82db-662f038b397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:19.354179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375382c-58b2-448a-82db-662f038b397c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:19.354601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375382c-58b2-448a-82db-662f038b397c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:19.354977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375382c-58b2-448a-82db-662f038b397c None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:19.357052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375382c-58b2-448a-82db-662f038b397c None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:19.358688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375382c-58b2-448a-82db-662f038b397c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:19.360311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:19.371672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 82/162] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:19 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:19.380962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375382c-58b2-448a-82db-662f038b397c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:19.429099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:19.557244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:19.634568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2baeaf-b1e8-42c5-ae69-4f52c2f7883b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:19.742750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375382c-58b2-448a-82db-662f038b397c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:19.791267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:19.792375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:19.792853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:19.793360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:19.795916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=c1f81d15d7c54a30853ee3ff06b25bdd) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:19.796461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:19.800693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b345c52-4b46-4d4c-94f2-f0f75b8ec550 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:19.820198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 81/163] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:19 2022] GET /identity/v3/role_assignments?user.id=c1f81d15d7c54a30853ee3ff06b25bdd&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:19.870847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:19.872497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:19.872982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:19.873454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:19.874983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:19.875456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:19.878094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8a10c-9dcf-4a07-a572-bfc526d39a79 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:19.896075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 83/164] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:19 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 608 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:20.289499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4645-c9e1-464a-9622-ef467214af58 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:20.290401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4645-c9e1-464a-9622-ef467214af58 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:20.290828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4645-c9e1-464a-9622-ef467214af58 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:20.292526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 82/165] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:20.302105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:20.302645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:20.303111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:20.350346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:20.352436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:20.503954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 84/166] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:20.547464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d4645-c9e1-464a-9622-ef467214af58 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:20.682456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:20.691628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d4645-c9e1-464a-9622-ef467214af58 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:20.763165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:20.764454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:20.764988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:20.765435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:20.788394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-58220fab-89ec-49b0-a2f6-b54f8a03352a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:20.804394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:20.805367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:20.806291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:20.806904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 83/167] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:20.815463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:20.837085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd1e1-25b0-4cbd-83b9-6ad35e74fb63 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:20.837574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd1e1-25b0-4cbd-83b9-6ad35e74fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:20.838058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cd1e1-25b0-4cbd-83b9-6ad35e74fb63 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:20.842695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 84/168] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:20.851386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:20.851843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:20.852306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:20.894755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:20.897521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:20.921806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:21.048131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1f7d7-7596-438d-a671-c5bb9c769a60 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:21.056800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 85/169] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:21.129030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:21.130645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:21.131090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:21.131320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:21.131539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:21.132805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:21.133296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:21.135856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:21.160186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 85/170] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:21.177697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:21.379337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:21.429925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6fe6b-bc08-4bff-a43a-1fc9a9339e46 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:21.552289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:21.553649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:21.554133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:21.554668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:21.566776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d92b4-1973-470f-a3f0-952bd6537b4c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:21.598813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:21.620784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-374904a8-6cac-402b-9c5e-e06c68dd4326 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:21.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:21.641580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:21.642492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:21.643101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 86/171] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:21.645622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646183-fecf-43ac-b702-721a7b66f0bc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:21.646077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646183-fecf-43ac-b702-721a7b66f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:21.654631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646183-fecf-43ac-b702-721a7b66f0bc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:21.657766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 87/172] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:21 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:21.659043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3646183-fecf-43ac-b702-721a7b66f0bc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:21.669613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:21.670966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:21.671488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:21.671966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:21.673778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:21.675681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f60bf-19bf-403c-9060-eeea8a913624 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:21.677779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 86/173] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:21 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:21.679373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:21.679868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:21.680355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:21.699365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:21.701580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:21.861027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 87/174] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:21.862637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:22.006148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3646183-fecf-43ac-b702-721a7b66f0bc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:22.110140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcacc4f-2a02-43dc-8846-0f10f1a85fc4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:22.139093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:22.139348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:22.140499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.140949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.141396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.142838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:22.143349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:22.145918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:22.178089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 88/175] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:22.179655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:22.199422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:22.202027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.202599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.203088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.226785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:22.432236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:22.435590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:22.437814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9412ddd-9aff-4ddd-8775-53c7b47c3934 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:22.507981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:22.509360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.509859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.510388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.531771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:22.556039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:22.557642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:22.558544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 89/176] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:22.560285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:22.717836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 88/177] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:22.717836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:22.725716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:22.730038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:22.766344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3301b9-f810-49a9-be9c-9740cb20e30a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:22.814618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:22.815551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.815803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.816065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] PATH_INFO: `/v3/users/f83fca0dd8e84a8191f04222fdc03da6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.817374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] RBAC: Authorizing `identity:get_user(user_id=f83fca0dd8e84a8191f04222fdc03da6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:22.818931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e4e26-055c-4bd0-ad56-fb8a83a2d29e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 90/178] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:22 2022] GET /identity/v3/users/f83fca0dd8e84a8191f04222fdc03da6 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79122dfb-38cc-4d31-8654-e14b7a9c47e4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79122dfb-38cc-4d31-8654-e14b7a9c47e4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79122dfb-38cc-4d31-8654-e14b7a9c47e4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 91/179] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:22.826491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a437d-fdc7-4b43-b5d7-c7afe0fc5121 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.827250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a437d-fdc7-4b43-b5d7-c7afe0fc5121 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.827250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a437d-fdc7-4b43-b5d7-c7afe0fc5121 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.828111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 92/180] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:22.828351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a437d-fdc7-4b43-b5d7-c7afe0fc5121 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:22.883533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c363-034e-481d-ad4c-b3859781dcae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:22.927470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:22.928299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.928593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.928883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.929841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:22.930173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:22.931706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d26de-9223-407b-ac6c-fecdfd93e751 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:22.951323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 89/181] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:22.951323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:22.951323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:22.951323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:22.968945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:22.970129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:23.036945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a437d-fdc7-4b43-b5d7-c7afe0fc5121 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.059702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 90/182] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:23.060908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.065974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.116439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.118703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.119184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.119681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.123486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:23.126088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65e9ee-6548-4101-8584-04a18491cbdb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:23.137044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 93/183] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:22 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:23.137044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.137044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.137044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.151307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.160882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:23.163098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:23.214851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81314f16-fcc1-4ff7-af2a-01892061cfaf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.255561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.256406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.256749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.257130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.266519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 94/184] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:23.268227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.351567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:23.353058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:23.356756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.407269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b63a-5e04-4ae0-979a-7e2b921917f0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.449163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.450288 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.450472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.450813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.453934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.467170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:23.469078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:23.471067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 95/185] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:23.472372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.498830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 91/186] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:23.500537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.610614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0bfcc-076a-4613-9097-e1107f0a797e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.643861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7db1-fd98-43d4-83a5-b005de6285fa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.647236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.655061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.656039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.656402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.656776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.669854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7834e9ba-8ee9-4bb1-a387-22b486766e1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:23.678248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:23.679270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:23.680051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 96/187] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:23.681231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12976-842a-4906-8c6a-ecb2a6a79a50 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.681231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12976-842a-4906-8c6a-ecb2a6a79a50 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.681536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12976-842a-4906-8c6a-ecb2a6a79a50 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.685126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 97/188] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:23.685126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12976-842a-4906-8c6a-ecb2a6a79a50 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.694762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.696072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.696516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.696959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.698442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:23.698950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:23.701269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:23.711515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 92/189] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:23.712868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.784205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:23.869450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12976-842a-4906-8c6a-ecb2a6a79a50 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.890009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2a3b-f7d9-4079-9866-42933691df42 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:23.929619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.930688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.931096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.932422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:23.933371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:23.937200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:23.939985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 98/190] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:23.941050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:23.955077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:23.956795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:23.957249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:23.957694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.021161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.075120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.077587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479e0ed-4aa9-499b-9133-e1174701c0dd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.104588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7463574-e4b7-4721-8560-ea117ddf9a3a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.140327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.156224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.157303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.157647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.158031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] PATH_INFO: `/v3/projects/0184a3906c2b47fab49e4aa9a77386ce` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.159534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] RBAC: Authorizing `identity:get_project(project_id=0184a3906c2b47fab49e4aa9a77386ce)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.161540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.163897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 99/191] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/projects/0184a3906c2b47fab49e4aa9a77386ce => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:24.165153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.223208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 93/192] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:24.225485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.225924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.226390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.247158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:24.248686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:24.312068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4701a-9258-4675-acfe-441691ebd1cc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.345321 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.351888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 94/193] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:24.352989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.370516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.371705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.372205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.372696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.374145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:24.374783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.377418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.399711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 100/194] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:24.399711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.468868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.473694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e61d7-3223-4cc0-954b-02087f2383f5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.516581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.517448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.517857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.518320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.519183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.520927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.528752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8e75e-53df-4429-a3a5-b1836ece3674 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.573171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.573900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.574241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.574581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.575449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0184a3906c2b47fab49e4aa9a77386ce, user.id=f83fca0dd8e84a8191f04222fdc03da6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:24.575768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.577273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.584163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 101/195] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/role_assignments?user.id=f83fca0dd8e84a8191f04222fdc03da6&scope.project.id=0184a3906c2b47fab49e4aa9a77386ce&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:24.585033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.629361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 95/196] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:24 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:24.631296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.745673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.784618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343654-7463-4ba6-a1c4-4df580ea1a27 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.825569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:24.833443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbace9e-d95a-4052-874d-b081230a2eff None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:24.861013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.862941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.863645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.864427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.866923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:24.869813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:24.873462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 102/197] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:24.875307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:24.920466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:24.921743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:24.922234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:24.922679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:24.940968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:24.957221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:24.958759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:24.960060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:24.964656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 96/198] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:24.964656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:25.076879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c7d2-fb1b-4d56-9b4c-174ee578105c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:25.147140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:25.148576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:25.149124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:25.149528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:25.151081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8879-aeb6-40eb-b3ea-3bd18434d85a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:25.151702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:25.154398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:25.157513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 103/199] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:25.159690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:25.169620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:25.205729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:25.207674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:25.208249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:25.208730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:25.210611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=a08520b93daa4e57ae62d5a346f11e36) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:25.211177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:25.213981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:25.227363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 97/200] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:24 2022] GET /identity/v3/role_assignments?user.id=a08520b93daa4e57ae62d5a346f11e36&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:25.228689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:25.261828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:25.365319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0175fc-e2dd-4756-8ae9-086f1bae499b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:25.472959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:25.474740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:25.475415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:25.476555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:25.477863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:25.480103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:25.482261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:25.500579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e284b4-dfb1-4302-a8a5-8ba701616935 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:25.511090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 104/201] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:25.519545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:25.570478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:25.571428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:25.571711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:25.571995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:25.686731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:25.723548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:25.725079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:25.745733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dedf3e-9c2f-458e-a5d8-4e68b07f1788 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:25.763449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:25.816187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:25.817084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:25.817360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:25.817627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] PATH_INFO: `/v3/users/85d4d1ec048049ac9071edf9e949d9d2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:25.819022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] RBAC: Authorizing `identity:get_user(user_id=85d4d1ec048049ac9071edf9e949d9d2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:25.820510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:25.823502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 105/202] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:25 2022] GET /identity/v3/users/85d4d1ec048049ac9071edf9e949d9d2 => generated 289 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:25.829494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:25.947603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 98/203] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:25.956637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:26.062431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:26.063600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c060f-9426-44fe-898a-3d59e2fe922a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:26.087087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d8df-bb4b-4935-83a5-19b92903c3a5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:26.128200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:26.129134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:26.129423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:26.129707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:26.132861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:26.133726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:26.134145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:26.134597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:26.135051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:26.136534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:26.142559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 99/204] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:25 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:26.150054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:26.203641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:26.220800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:26.222661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:26.362942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471dec-e6b2-41ec-a1da-f46cdbad4e8f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:26.411311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:26.411993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:26.412275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:26.412551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:26.413419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=a08520b93daa4e57ae62d5a346f11e36) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:26.413734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:26.415273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:26.429618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 100/205] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:26 2022] GET /identity/v3/role_assignments?user.id=a08520b93daa4e57ae62d5a346f11e36&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:26.448239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 106/206] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:26.454769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:26.471522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:26.601380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecc5a-89f2-4db2-bf1f-f5e1a1780286 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:26.664910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:26.665809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:26.666095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:26.666402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:26.671231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:26.672787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39381ea-a853-4fbd-ac5a-55ab4f64f289 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:26.674975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 101/207] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:26 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:26.680932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:26.807959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:26.834704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c53d-8b7a-4d64-88bb-c98293ca43de None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:26.888958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:26.889634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:26.889916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:26.890205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:26.891068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, scope.project.id=06620fd9d314432ab7107312bc93402f, user.id=85d4d1ec048049ac9071edf9e949d9d2) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:26.891382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:26.892875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa3e36-6f2e-4492-9fa6-5f43e3d3ab70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:26.904083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 107/208] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:26 2022] GET /identity/v3/role_assignments?user.id=85d4d1ec048049ac9071edf9e949d9d2&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:27.755293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6d14-ceda-4698-acb8-fef21f5db34b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:27.755835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6d14-ceda-4698-acb8-fef21f5db34b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:27.755835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad6d14-ceda-4698-acb8-fef21f5db34b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:27.757380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 102/209] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:27.764216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22206c4-052b-4ee3-b222-edc14ce40c92 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:27.764721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22206c4-052b-4ee3-b222-edc14ce40c92 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:27.764793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22206c4-052b-4ee3-b222-edc14ce40c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:27.783964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e22206c4-052b-4ee3-b222-edc14ce40c92 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:27.785194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e22206c4-052b-4ee3-b222-edc14ce40c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:27.871739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 108/210] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:27.892833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad6d14-ceda-4698-acb8-fef21f5db34b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:27.952874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db83203-c4ac-4abf-945d-05ec91829083 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:27.953192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db83203-c4ac-4abf-945d-05ec91829083 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:27.953438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db83203-c4ac-4abf-945d-05ec91829083 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:27.954889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 109/211] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:27.961137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:27.961462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:27.961697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:27.984776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:27.987177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:28.007552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:28.034999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad6d14-ceda-4698-acb8-fef21f5db34b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.076746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.077940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.078463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.078911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.080501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:28.081720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 110/212] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:28.087005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:28.093881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 103/213] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:28.095639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:28.105318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:28.106345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:28.222292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d80e2-5110-409d-bfe7-089a35785248 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.240099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ee09c-075c-4f77-ba81-7db2c8b90c14 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.261726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.262878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.263360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.263826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.264762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:28.266480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:28.287314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.288217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.288504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.288825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.302065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:28.310476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:28.311401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:28.312199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 104/214] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:28.319269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:28.329273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:28.382607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 111/215] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:28 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:28.435716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98d6ba-492c-4458-9c08-572c0d13afb5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.477336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.477935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.478244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.478530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.479318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:28.479623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:28.481979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:28.491996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 105/216] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:28.498552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:28.577447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:28.625446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939c10-64c2-4107-ab36-b8d03d21d414 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.667752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.669437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.669715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.669989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] PATH_INFO: `/v3/users/7ce493418ef94f32af6bb9f950424df6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.687624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] RBAC: Authorizing `identity:get_user(user_id=7ce493418ef94f32af6bb9f950424df6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:28.689574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7970346-32ca-4717-93d8-210f54e5533e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:28.696964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 112/217] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:28 2022] GET /identity/v3/users/7ce493418ef94f32af6bb9f950424df6 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:28.698664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:28.776889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:28.815013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99136-9d35-43d5-9c95-c1e409e14644 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:28.856621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:28.857650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.858220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.858712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.875536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad5b8-76e7-464d-91d5-78e6d647bcba None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.876593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad5b8-76e7-464d-91d5-78e6d647bcba None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.876705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dad5b8-76e7-464d-91d5-78e6d647bcba None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.878311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 113/218] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:28.884816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:28.884816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:28.884816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:28.905554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:28.906783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:28.960023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:28.961513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:28.999628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 114/219] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:29.022048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.044399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:29.089534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:29.098547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 106/220] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:29.104122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.182517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1501ad5-fea6-4d09-a7cd-2decaea744b6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:29.222034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:29.222887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.223163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.223430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.235415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:29.244004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:29.244947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:29.245731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 115/221] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:29.251227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.365875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fc7e-2179-4936-869e-049bdcae1da5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:29.378994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:29.404132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f5e5b-690b-46a2-89cc-c36a5a26d7b0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:29.409838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:29.410540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.410817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.411090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.411848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:29.412155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:29.414181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:29.431729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 116/222] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:29.439699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.481697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:29.482829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.483590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.483868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.485164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:29.486738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217a0d-5c4b-455b-9936-4dfd64d72d94 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:29.489762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 107/223] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:29.491525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76d114-fd31-467d-8505-db5454763584 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.491817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76d114-fd31-467d-8505-db5454763584 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.492087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76d114-fd31-467d-8505-db5454763584 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.493989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 108/224] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:29.497536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76d114-fd31-467d-8505-db5454763584 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.546059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:29.644999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76d114-fd31-467d-8505-db5454763584 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:29.671317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bf417-6f5a-4553-8d36-eee8cf55e0c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:29.711401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:29.712619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.713038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.713353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:29.713886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.714714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.715273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.715710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:29.715876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.717483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8009b5-2c13-49a6-af93-b1ec32711d79 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:29.717611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=7ce493418ef94f32af6bb9f950424df6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:29.718916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:29.719673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 117/225] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:29.721100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:29.721551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:29.721964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:29.722323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:29.733801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 109/226] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/role_assignments?user.id=7ce493418ef94f32af6bb9f950424df6&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:29.735636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.740868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:29.742631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:29.814721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:29.932406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 118/227] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:29.960685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:29.998967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:30.004116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d5a8-1a26-4357-bb5e-20f0391496ee None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.073305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.074899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.075568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.076080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.078377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:30.081096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:30.084629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 110/228] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:30.096118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:30.104960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247b6d0-e131-4542-9960-9fe172cfa3a5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.145099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.145922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.146222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.146525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.159276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:30.167351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:30.169097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:30.170088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:30.171160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 119/229] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:30.176876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:30.211295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:30.290426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef99cb-22a8-4cf9-aa78-98cca4b72baa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.334244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.334244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.334244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.334244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.334960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:30.334960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:30.335230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:30.351106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 120/230] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:30.354254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:30.400912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689dd9-b697-4022-bb9b-0a8b9108d5ef None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.480202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.481119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.481402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.481681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/a08520b93daa4e57ae62d5a346f11e36/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.486245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, user_id=a08520b93daa4e57ae62d5a346f11e36, role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:30.486245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:30.519961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:30.530551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490475f4-5a5b-48a7-b842-dc5d55ca6b83 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.564570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 111/231] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:30 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/a08520b93daa4e57ae62d5a346f11e36/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:30.585962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.586897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.587179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.587455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.588809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:30.591272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:30.597930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 121/232] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:30 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:30.599490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:30.637845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:30.763999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8cdb-cff5-465a-86bd-8a3d2a0ea21b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:30.826423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:30.827869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:30.828377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:30.828882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:30.831027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:30.833765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf49f-968b-4741-95f9-f0c73afe5bd1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:30.837010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 112/233] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:30 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:30.842924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:30.977851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bacbea-efa6-44e2-af35-d31277e4866e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:31.040279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:31.041432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.041857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.042305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/a08520b93daa4e57ae62d5a346f11e36/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.047966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, user_id=a08520b93daa4e57ae62d5a346f11e36, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:31.049771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbfc79-d0e1-4a80-8584-b2fafd205f94 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:31.181376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 122/234] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:30 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/a08520b93daa4e57ae62d5a346f11e36/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:31.373150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858d332-cb0b-4bba-bcc9-126448bd24e2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.373652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858d332-cb0b-4bba-bcc9-126448bd24e2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.373990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858d332-cb0b-4bba-bcc9-126448bd24e2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.376784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 113/235] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:31.393174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80945-94aa-42ca-8dd0-6994668b6fef None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.393757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80945-94aa-42ca-8dd0-6994668b6fef None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.394277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80945-94aa-42ca-8dd0-6994668b6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.423470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f9c80945-94aa-42ca-8dd0-6994668b6fef None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:31.425057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f9c80945-94aa-42ca-8dd0-6994668b6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:31.445225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:31.520026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 123/236] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:31.552392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857806-7163-4e44-b706-230d8171e7af None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.552894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857806-7163-4e44-b706-230d8171e7af None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.553426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857806-7163-4e44-b706-230d8171e7af None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.556593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 114/237] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:31.564211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.564539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.564916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.572640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95857806-7163-4e44-b706-230d8171e7af None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:31.588781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:31.590981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:31.741667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 124/238] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:31.789134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:31.795331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:31.800337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95857806-7163-4e44-b706-230d8171e7af None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:31.847955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:31.848832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:31.849245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:31.849663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:31.870052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:31.884518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:31.885462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:31.886275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 115/239] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:31.892063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:31.936230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:31.958124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d90cf-91e2-420c-a3c7-f57f45754cc2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:32.017489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:32.018318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:32.018602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:32.018876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:32.021407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:32.022903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:32.024559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 125/240] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:32.033677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:32.133809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af70fb-f714-4b96-8255-5199a1a8d8f9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:32.205084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:32.206546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:32.207183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:32.207858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:32.209411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:32.210095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:32.222232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:32.244609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 116/241] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:32.250815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:32.288833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441d2e-ce03-4da7-9e4f-bf7d375c7e2e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:32.308086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:32.357851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:32.359187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:32.359749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:32.360305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:32.361832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:32.364794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:32.504111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 126/242] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:32 2022] POST /identity/v3/users => generated 293 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:32.509716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fe565-5050-4153-b59a-1b15544e8ac8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:32.611708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:32.613085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:32.613551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:32.614012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] PATH_INFO: `/v3/users/23f0f8c2e3a141d7a08863fe12f988c6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:32.651518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] RBAC: Authorizing `identity:get_user(user_id=23f0f8c2e3a141d7a08863fe12f988c6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:32.654527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:32.657878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 117/243] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:32 2022] GET /identity/v3/users/23f0f8c2e3a141d7a08863fe12f988c6 => generated 297 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:32.664003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:32.674600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:32.838327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca645c-38fe-4799-8a01-7c4f2a3f03a5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:32.891138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:32.891835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:32.892115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:32.892389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:32.972076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:32.973621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c28c7d-c0cc-4bdf-a0f7-eec90bb38418 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:32.986770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:33.086008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 127/244] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:14:33.092761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:33.113009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478a29-fbc4-4f3d-adb9-6967f4ee0e45 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.113335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478a29-fbc4-4f3d-adb9-6967f4ee0e45 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.114026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9478a29-fbc4-4f3d-adb9-6967f4ee0e45 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.116036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 128/245] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:33.127207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.128092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.128377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.152244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:33.154027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:33.258014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 129/246] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:33.294852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:33.305682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10f6f3-064d-4f56-8a19-7ca34d9a691a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:33.307930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:33.358016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:33.386551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:33.388094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.388602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.389099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.391420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:33.394132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:33.399218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 118/247] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:33.406784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:33.462953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c99a5e-739f-4a0d-8d12-c66520a5d9d8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:33.516033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:33.516863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.517137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.517404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.531444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:33.540599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:33.543259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:33.545260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 130/248] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:33.554231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:33.637490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67167b-2961-4b2a-b043-3d25ffb2fa90 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:33.680326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:33.689072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea497a63-3a12-4e2f-b924-4fd004da27a6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:33.711849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:33.713092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.713587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.714066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.715752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=23f0f8c2e3a141d7a08863fe12f988c6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:33.716994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:33.718538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:33.728106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 119/249] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/role_assignments?user.id=23f0f8c2e3a141d7a08863fe12f988c6&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:33.735947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:33.736651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.736930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.737203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.737998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:33.738333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:33.739827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:33.751149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 131/250] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:33.757457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:33.776770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:33.909250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508bb59-4672-4547-a7f1-af6aeb73ac63 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:33.961221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:33.962113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:33.962414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:33.962677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] PATH_INFO: `/v3/users/7ce493418ef94f32af6bb9f950424df6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:33.963914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] RBAC: Authorizing `identity:get_user(user_id=7ce493418ef94f32af6bb9f950424df6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:33.965464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:33.971646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 120/251] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/users/7ce493418ef94f32af6bb9f950424df6 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:33.973269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:34.110981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedccd-fe19-4057-bfca-7ec4aedf92ae None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:34.149591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:34.150509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.150790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.151064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.152427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:34.154650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b092c-7f10-4f6a-a971-3d66fb58231d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:34.160972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 132/252] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:33 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:34.163086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:34.218936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cb6b3-fc64-4ec5-aa50-c1e4b1957aa3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.219240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cb6b3-fc64-4ec5-aa50-c1e4b1957aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.219545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cb6b3-fc64-4ec5-aa50-c1e4b1957aa3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.221128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 133/253] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:34.226520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.226880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.227056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.257451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:34.264492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:34.304924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:34.333690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:34.405254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b79888-519b-407e-8de6-ce233ad0580e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:34.410235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 134/254] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:34.447687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:34.488576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:34.490091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.490616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.491102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/7ce493418ef94f32af6bb9f950424df6/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.494847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, user_id=7ce493418ef94f32af6bb9f950424df6, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:34.497714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a4fd4-a972-4d14-aa6e-fbfc67259707 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:34.623332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18f9f4-86e2-4b5c-b4dd-b0d250177564 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:34.688981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:34.690290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.690770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.691249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.713605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-2956b56f-f6c4-4e6c-9df0-bc6cbba2bf2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:34.729951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:34.731040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:34.731863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:34.732507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 135/255] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:34.734261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5b32-d84a-4362-a08f-8131e0153cd9 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.734728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5b32-d84a-4362-a08f-8131e0153cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.738226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5b32-d84a-4362-a08f-8131e0153cd9 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.738226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 136/256] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:34.743843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5b32-d84a-4362-a08f-8131e0153cd9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:34.774709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:34.797037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 121/257] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:34 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/7ce493418ef94f32af6bb9f950424df6/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:34.798478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:34.798761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:34.799036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:34.838671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:34.839893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:34.966182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5b32-d84a-4362-a08f-8131e0153cd9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:35.034329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:35.035393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:35.035862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:35.036326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:35.036796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 122/258] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:35.037818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:35.038387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:35.040991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:35.059134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 137/259] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:35.064988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.077953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.205480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:35.272540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcca2b-0d24-408e-9dd3-3e257849c6eb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:35.276137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:35.311302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6b7c7-1825-4151-ad12-ee4ea34912d4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:35.325700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:35.326612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:35.326887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:35.327156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:35.328392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:35.329898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:35.335767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 123/260] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:35.342016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.381513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:35.383197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:35.383694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:35.384181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:35.405890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:35.424287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:35.426504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:35.428607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 138/261] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:35.436262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.485830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f0259-60a1-462e-8961-10d8b84e42d0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:35.553243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:35.554088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:35.554406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:35.554678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:35.560659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:35.658803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810a990-4cfd-4202-ba6a-e121d94476c4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:35.710793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:35.711940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:35.732129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:35.733795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:35.734553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:35.735277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:35.737495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:35.738341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:35.741471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:35.754823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:35.760989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 139/262] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:35.770542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.869334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 124/263] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:14:35.875120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:35.944600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:36.052174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7008f-ba06-4d54-ae92-ae794a5bcd8c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:36.114449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a597b91-5ffb-4d65-be68-961391b18e23 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:36.143130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:36.143927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.144208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.144499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] PATH_INFO: `/v3/users/a08520b93daa4e57ae62d5a346f11e36` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.145801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] RBAC: Authorizing `identity:get_user(user_id=a08520b93daa4e57ae62d5a346f11e36)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:36.147421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:36.149460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 140/264] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/users/a08520b93daa4e57ae62d5a346f11e36 => generated 283 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:36.154983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:36.201745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:36.202665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.202944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.203201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.204422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:36.205883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:36.212459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 125/265] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:35 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:36.217877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:36.335583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:36.385755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e9d36-c852-44ea-9b2d-f9bcc4b6381d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:36.403492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:36.447662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1bb39-709e-476c-ba97-c48f5e51676b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:36.454960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:36.455682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.455965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.456238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.457098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=a08520b93daa4e57ae62d5a346f11e36) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:36.457410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:36.458919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2997f8f-3b42-42cd-97c4-4e9ab0ed8759 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:36.469845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 126/266] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:36 2022] GET /identity/v3/role_assignments?user.id=a08520b93daa4e57ae62d5a346f11e36&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:36.533701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:36.534945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.535377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.535812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.708126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:36.711181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:36.836906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c2e1-a7ca-4654-aadc-775e0a41055a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.837252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c2e1-a7ca-4654-aadc-775e0a41055a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.837458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c2e1-a7ca-4654-aadc-775e0a41055a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.839152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 127/267] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:36.848209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c945b-9776-4441-8fb7-7e17ffd30ca4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:36.848855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c945b-9776-4441-8fb7-7e17ffd30ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:36.849095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:36.849533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c945b-9776-4441-8fb7-7e17ffd30ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:36.892099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1c7c945b-9776-4441-8fb7-7e17ffd30ca4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:36.896324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1c7c945b-9776-4441-8fb7-7e17ffd30ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:36.949524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 141/268] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:14:36.955681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.067758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 128/269] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:37.069399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f01644-7345-4465-916f-7d4bd98d8444 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.069685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f01644-7345-4465-916f-7d4bd98d8444 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.070058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f01644-7345-4465-916f-7d4bd98d8444 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.072023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 129/270] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:37.079777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.080094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.080324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.107316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:37.108914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:37.149591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b61f-dcac-4de0-8f46-3258a34d92c6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.186344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 130/271] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:37.187716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.219828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.221394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.221980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.222873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.225634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:37.228925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:37.231911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:37.233969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 142/272] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:36 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:37.243018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.312551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:37.320424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426af1-9bab-40cc-931b-92a842193a66 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.362925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.365614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.366159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.366665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.368161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:37.369885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:37.377864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 131/273] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:37.379294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.385848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628f0b-1def-4f5c-a770-5e7b1a0fda6e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.436212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.437487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.437979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.438506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.456316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:37.464311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:37.466083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 143/274] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:37.468060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.492906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:37.501139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84f775-10e9-49b3-b0da-b11d6b72d0d1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.543958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.545243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.545986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.546290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.547330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=a08520b93daa4e57ae62d5a346f11e36) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:37.547645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:37.549151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4afe2-ce55-4984-a5e0-a39aa000e24b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:37.556010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 132/275] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/role_assignments?user.id=a08520b93daa4e57ae62d5a346f11e36&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:37.557376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992fac-9f4d-41b4-89e5-f3dbca92b948 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.557664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992fac-9f4d-41b4-89e5-f3dbca92b948 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.557933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992fac-9f4d-41b4-89e5-f3dbca92b948 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.564487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 133/276] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:37.564487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a992fac-9f4d-41b4-89e5-f3dbca92b948 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.607464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:37.615542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897647b-67de-41de-96f7-49372f9bb5cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.674430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.675152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.675485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.675760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.676503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:37.679253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:37.703858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a992fac-9f4d-41b4-89e5-f3dbca92b948 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:37.748965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:37.749650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.749928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.750216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.750976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:37.751278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:37.752716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adbf0f-df3f-41a2-aa8c-393715d41442 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:37.766796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 134/277] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:37.766796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:37.766796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:37.766796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:37.783439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:37.784641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:37.792099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 144/278] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:14:37 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:14:37.792977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.803693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:37.876635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 135/279] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:37.900569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:37.961169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e9bf8-4a3a-40db-bdb6-eae579afceb8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.004962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.006408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.006926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.007417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] PATH_INFO: `/v3/users/989dbb23d61c430ba0939864cceaa07c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.024553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] RBAC: Authorizing `identity:get_user(user_id=989dbb23d61c430ba0939864cceaa07c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:38.026520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:38.029179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 145/280] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/users/989dbb23d61c430ba0939864cceaa07c => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:38.030654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670f4ec-720d-4f7a-bada-c6ad6a0b1a28 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.036837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:38.054107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:38.085013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.086095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.087394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.087833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.105677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:38.114359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:38.115474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:38.116249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:38.121860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 136/281] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:38.123486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:38.141732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:38.186290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47cbb2-49bc-45e5-978c-a1b46c3dc209 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.235602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.236281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.236571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.236835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.237677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, user.id=989dbb23d61c430ba0939864cceaa07c, scope.system=all) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:38.237991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:38.239499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:38.247646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 146/282] 10.222.0.48 () {56 vars in 1308 bytes} [Fri May 27 09:14:38 2022] GET /identity/v3/role_assignments?user.id=989dbb23d61c430ba0939864cceaa07c&scope.system=all&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:38.295024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8a05b-7655-418c-926f-994c459469d8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.348908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.349623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.349903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.350179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.351032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:38.351338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:38.352817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb17c8-064e-4f7d-b36e-0594eee6bafb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:38.368535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 137/283] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:38.375366 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:38.506453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:38.553046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6069f8-41ac-4846-ab23-3da414c2ee86 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.553684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6069f8-41ac-4846-ab23-3da414c2ee86 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.554191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6069f8-41ac-4846-ab23-3da414c2ee86 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.556066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 138/284] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:38.562470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.563013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.563492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.576896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79280aea-de83-4277-b195-4bde8d004958 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.602418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:38.603631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:38.654490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.656007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.656457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.656829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] PATH_INFO: `/v3/users/23f0f8c2e3a141d7a08863fe12f988c6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.659659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] RBAC: Authorizing `identity:get_user(user_id=23f0f8c2e3a141d7a08863fe12f988c6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:38.662421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:38.666783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 147/285] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:38 2022] GET /identity/v3/users/23f0f8c2e3a141d7a08863fe12f988c6 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:38.677513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:38.773102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 139/286] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:38.797650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:38.800145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:38.894145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5bfde-03c8-499e-aea7-798f19e773c5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:38.975761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:38.977183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:38.977640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:38.978098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:38.984547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:38.987153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:38.990297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 148/287] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:38 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:38.999402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:39.003486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8a9c3-245a-4741-b425-736b78e7d9ca None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:39.021259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:39.062529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:39.066235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:39.066235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:39.066235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:39.085681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:39.101584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:39.102595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:39.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:39.104474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 140/288] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:39.111622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:39.232372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def493f4-b494-49a2-97b5-80e6d78b2893 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:39.310571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:39.347638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:39.348460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:39.348744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:39.349025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/23f0f8c2e3a141d7a08863fe12f988c6/roles/7e5337e63b0b4809b748baf1e1396bf6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:39.357032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, user_id=23f0f8c2e3a141d7a08863fe12f988c6, role_id=7e5337e63b0b4809b748baf1e1396bf6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:39.358684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:39.384285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa362-d324-44b3-9a19-3f7dbf1090cb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:39.426168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:39.426914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:39.427195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:39.427538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:39.428387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:39.428873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:39.430462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:39.441225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 141/289] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:39.447427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:39.462167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 149/290] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:14:38 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/users/23f0f8c2e3a141d7a08863fe12f988c6/roles/7e5337e63b0b4809b748baf1e1396bf6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:39.650696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3615-4e65-491a-96fe-d3f0a100b306 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:39.684662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:39.701073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:39.701972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:39.702274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:39.702549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] PATH_INFO: `/v3/users/7ce493418ef94f32af6bb9f950424df6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:39.704034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce493418ef94f32af6bb9f950424df6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:39.705599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:39.707749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 142/291] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:39 2022] GET /identity/v3/users/7ce493418ef94f32af6bb9f950424df6 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:39.713273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:39.819015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:39.860696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f404b-9e3a-41a9-aa52-9ce3c45fb32e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:39.899566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:39.900225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:39.900507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:39.900853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:39.983391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:39.984883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:40.062419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:40.105646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 150/292] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:40.111614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:40.259825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945da0-db09-49ea-9088-b3c513c5f4f3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:40.312297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:40.313232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:40.313516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:40.313789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:40.315066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:40.316552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f820339-0854-43dd-9a87-210d6c17965d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:40.318575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 143/293] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:40 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:40.325095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:40.479531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b2ac8-bfbb-481c-867c-0f817147c5f8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:40.524047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:40.524634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:40.524915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:40.525192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:40.526048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=7ce493418ef94f32af6bb9f950424df6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:40.526399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:40.528033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6ea13-7fc3-4c28-8165-6b942e245c91 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:40.535155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 151/294] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:40 2022] GET /identity/v3/role_assignments?user.id=7ce493418ef94f32af6bb9f950424df6&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:40.908901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56891-04b4-4f6d-801b-c9a2925eb072 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:40.909135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56891-04b4-4f6d-801b-c9a2925eb072 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:40.909507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56891-04b4-4f6d-801b-c9a2925eb072 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:40.911067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 144/295] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:40.917534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:40.918980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:40.918980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:40.940695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:40.941903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:40.979806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.010676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c9d0e-624c-42b5-8616-307bf8e33142 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.011105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c9d0e-624c-42b5-8616-307bf8e33142 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.011324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c9d0e-624c-42b5-8616-307bf8e33142 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.012986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 145/296] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:41.018777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.019177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.019400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.045860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:41.047116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:41.048410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 152/297] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:41.071438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.138939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.154416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 146/298] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:41.174246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.192277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed958b4-f8c1-497c-b99d-f8f4171aacfe None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.234534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.235335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.235613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.235863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.247689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:41.257502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:41.260480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:41.263598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 153/299] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:41.271823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.298214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8a2c2-b240-4f2f-a97b-57a03ac19b36 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.314399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.357875 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.359106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.359590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.360086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.361563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:41.363304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:41.365518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 147/300] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:41.371498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.426832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7889de-a145-4a85-91e2-e1821c521175 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.453919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.479275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.480905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.481733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.482263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.483407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:41.485499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:41.487911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:41.505196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 154/301] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:41.506420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.513038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b75bf2-82e9-4be4-889d-3d57f2e831b1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.554277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.555437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.555972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.556476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] PATH_INFO: `/v3/groups` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.557427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:41.559398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bfe05-6ee7-470d-8252-f252bf396846 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:41.615892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.634100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf45dea-ee2f-49c8-8c5c-9bd2d440811c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.638050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 148/302] 10.222.0.48 () {60 vars in 1134 bytes} [Fri May 27 09:14:41 2022] POST /identity/v3/groups => generated 229 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:14:41.640528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d1d0-b29b-41cd-8861-5d221b5cbca8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.640866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d1d0-b29b-41cd-8861-5d221b5cbca8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.641150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d1d0-b29b-41cd-8861-5d221b5cbca8 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.642892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 149/303] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:41.649591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.650327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.650694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.681396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:41.682649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:41.689308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.690734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.691270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.691759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] PATH_INFO: `/v3/users/989dbb23d61c430ba0939864cceaa07c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.694351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] RBAC: Authorizing `identity:get_user(user_id=989dbb23d61c430ba0939864cceaa07c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:41.697321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:41.700922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 155/304] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:41 2022] GET /identity/v3/users/989dbb23d61c430ba0939864cceaa07c => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:41.707270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.798229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 150/305] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:41.816615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:41.853331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9d4fd-2e63-4c2d-a6aa-377b7a3e7e55 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:41.874928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:41.905447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:41.906972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:41.908011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:41.910716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] PATH_INFO: `/v3/system/users/989dbb23d61c430ba0939864cceaa07c/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:41.912970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=989dbb23d61c430ba0939864cceaa07c, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:41.914614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:41.947440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1589a7-8e9f-4627-b850-07d390ca1294 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:42.000782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:42.002539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:42.003192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:42.003910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:42.018347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 156/306] 10.222.0.48 () {58 vars in 1334 bytes} [Fri May 27 09:14:41 2022] PUT /identity/v3/system/users/989dbb23d61c430ba0939864cceaa07c/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:42.029212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:42.037480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:42.039543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:42.041107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 151/307] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:42.053486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:42.086995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:42.185626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114aaba-ad07-430d-8e8a-99fa425f0ac9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:42.228087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:42.228682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:42.228964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:42.229241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:42.230017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:42.230370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:42.232453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:42.243142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 157/308] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:42.252623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:42.397646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075ea09-c532-4fde-b510-6c0324aebc17 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:42.409969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:42.442015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:42.443011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:42.443291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:42.443580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] PATH_INFO: `/v3/users/f94fb4b939e34cbe91be7a50894a0d70` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:42.460989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] RBAC: Authorizing `identity:get_user(user_id=f94fb4b939e34cbe91be7a50894a0d70)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:42.462654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac495bf-6776-4a22-8809-ed7938e99a75 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:42.465424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 152/309] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:42 2022] GET /identity/v3/users/f94fb4b939e34cbe91be7a50894a0d70 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:42.474486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:42.602924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:42.635664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f5e7a-c663-4c17-9b4e-9881fc3dcf70 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:42.702676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:42.703801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:42.704300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:42.704795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:42.706565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, user.id=f94fb4b939e34cbe91be7a50894a0d70, scope.system=all) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:42.707132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:42.709971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1762bbd-062d-4325-ac41-d740c95acafe None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:42.720883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 158/310] 10.222.0.48 () {56 vars in 1308 bytes} [Fri May 27 09:14:42 2022] GET /identity/v3/role_assignments?user.id=f94fb4b939e34cbe91be7a50894a0d70&scope.system=all&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:42.991844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304a9a-6a31-4f2b-a96e-178586747688 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:42.992073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304a9a-6a31-4f2b-a96e-178586747688 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:42.992448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304a9a-6a31-4f2b-a96e-178586747688 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:42.993738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 153/311] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:43.000443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:43.001138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:43.001633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:43.020505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:43.022576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:43.148009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 159/312] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:43.172664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55304a9a-6a31-4f2b-a96e-178586747688 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:43.308523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55304a9a-6a31-4f2b-a96e-178586747688 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:43.351862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:43.352936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:43.353366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:43.353811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:43.371393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:43.382294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:43.383431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:43.384404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:43.386531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 154/313] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:43.394945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:43.477109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:43.512332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2851c3-4d35-4574-a985-ba650e0f1125 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:43.553312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:43.554335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:43.554799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:43.555278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:43.556213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:43.556701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:43.558316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:43.568217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 160/314] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:43.575273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:43.606931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:43.693419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e6ad-3089-47ba-9284-21dd24be62a0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:43.734399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:43.735277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:43.735559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:43.735833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] PATH_INFO: `/v3/users/23f0f8c2e3a141d7a08863fe12f988c6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:43.737057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] RBAC: Authorizing `identity:get_user(user_id=23f0f8c2e3a141d7a08863fe12f988c6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:43.738527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:43.740669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 155/315] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:43 2022] GET /identity/v3/users/23f0f8c2e3a141d7a08863fe12f988c6 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:43.745987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:43.873911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f494547-331e-4552-8165-e64f5f3d9c54 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:43.916516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:43.917591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:43.918153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:43.918745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:43.922414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:44.005639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:44.007498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:44.164081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 161/316] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:44.170738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:44.354387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:44.367177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7bfb-cafb-46f3-9dd5-c67a713d3a06 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:44.437363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:44.439029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:44.439578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:44.440070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:44.442606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:44.445421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354e64-0c4c-4bc7-ab8b-dcf067905e46 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:44.449638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 156/317] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:44 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:44.457334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:44.492639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:44.574729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ea0de-7dfc-4665-917e-3006b33fd038 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:44.614376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:44.615040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:44.615322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:44.615599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:44.616435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, scope.project.id=0d776cda9a8749df88d778d848b78e4e, user.id=23f0f8c2e3a141d7a08863fe12f988c6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:44.616752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:44.618233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf05b18-1d16-4098-aba3-366de3bbef67 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:44.625224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 162/318] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:14:44 2022] GET /identity/v3/role_assignments?user.id=23f0f8c2e3a141d7a08863fe12f988c6&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:44.748301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624ecb-ea19-4e48-88a9-b609c72c4cc0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:44.749242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624ecb-ea19-4e48-88a9-b609c72c4cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:44.750005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624ecb-ea19-4e48-88a9-b609c72c4cc0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:44.753544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 157/319] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:44.759083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:44.759497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:44.759646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:44.782393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:44.783675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:44.860092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 163/320] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:44.883551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7624ecb-ea19-4e48-88a9-b609c72c4cc0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:45.012963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7624ecb-ea19-4e48-88a9-b609c72c4cc0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:45.052255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:45.053059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:45.053344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:45.053615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:45.054858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:45.056322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f51a6-8075-4afe-9852-3957f018d674 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:45.058250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 158/321] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:45.065212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:45.097868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:45.173926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d512-fc80-43a1-b001-6d0d454f4144 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:45.209528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:45.210225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:45.210506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:45.210782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] PATH_INFO: `/v3/groups` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:45.211472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:45.212934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34984a-b5cf-4f59-969f-7b5f1c731cf9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:45.336390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 164/322] 10.222.0.48 () {60 vars in 1134 bytes} [Fri May 27 09:14:45 2022] POST /identity/v3/groups => generated 226 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:14:45.631757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b1fd-c137-43c2-a9a9-c4a7ab0e44a7 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:45.632471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b1fd-c137-43c2-a9a9-c4a7ab0e44a7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:45.632628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b1fd-c137-43c2-a9a9-c4a7ab0e44a7 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:45.635274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 159/323] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:45.641627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b918-b5c8-401d-b479-cd50d102b58a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:45.642350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b918-b5c8-401d-b479-cd50d102b58a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:45.642856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b918-b5c8-401d-b479-cd50d102b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:45.665149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-be63b918-b5c8-401d-b479-cd50d102b58a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:45.667125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be63b918-b5c8-401d-b479-cd50d102b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:45.801279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 165/324] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:45.825420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037b1fd-c137-43c2-a9a9-c4a7ab0e44a7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:45.864597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:45.958573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037b1fd-c137-43c2-a9a9-c4a7ab0e44a7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.002614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.003424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.003697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.003965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.009062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433f5f-753f-4232-8364-8ffd5504984f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.009568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433f5f-753f-4232-8364-8ffd5504984f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.010062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433f5f-753f-4232-8364-8ffd5504984f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.013494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 166/325] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:46.018175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:46.023552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.024303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.024989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:46.026465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:46.028957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:46.030898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:46.033451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 160/326] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:46.034559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:46.055904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:46.057437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:46.076149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:46.196824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 167/327] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:46.243062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:46.264310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:46.274345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e5c8-e8cc-43f7-988e-abe5c9cb01b9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.313115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.313420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.313554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.313883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.314762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:46.315070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:46.316492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:46.331379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 161/328] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:46.333313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:46.470171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95fd9d-9e5f-4b76-90b0-214e26433c58 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.491417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601fcb8-a01e-4904-a251-c122a756eab9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.514126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.516079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.516079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.516079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] PATH_INFO: `/v3/users/989dbb23d61c430ba0939864cceaa07c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.517355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] RBAC: Authorizing `identity:get_user(user_id=989dbb23d61c430ba0939864cceaa07c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:46.519306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:46.526104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 162/329] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/users/989dbb23d61c430ba0939864cceaa07c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:46.528862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:46.563289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:46.581434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.581434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.581434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.581434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.606164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:46.615213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:46.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:46.616957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:46.619529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 168/330] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:46.628134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:46.653776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:46.729329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc159b1-617a-4c6e-81a6-b8f586e4ab57 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.794161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.795708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.796188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.796656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.797728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] RBAC: Adding query filter params (role.id=8ff134ce9e0f40f38af26ff674f3b83f, user.id=989dbb23d61c430ba0939864cceaa07c, scope.system=all) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:46.798189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:46.801103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:46.836114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 163/331] 10.222.0.48 () {56 vars in 1308 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/role_assignments?user.id=989dbb23d61c430ba0939864cceaa07c&scope.system=all&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 510 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:46.890638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f04f-81ae-4b81-b0a5-0e16b09b5ab0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:46.962354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:46.963368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:46.963793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:46.964313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:46.965886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:46.966546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:46.968598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:46.983075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 169/332] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:46.991226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:47.117378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:47.156370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f62f26-78c6-4603-b6a8-4d30da51a9b3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:47.220201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:47.221726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:47.222271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:47.222777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] PATH_INFO: `/v3/users/f94fb4b939e34cbe91be7a50894a0d70` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:47.224430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] RBAC: Authorizing `identity:get_user(user_id=f94fb4b939e34cbe91be7a50894a0d70)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:47.226691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da190775-30dc-4bfd-a2d5-b792c29caa82 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:47.230541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 164/333] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:46 2022] GET /identity/v3/users/f94fb4b939e34cbe91be7a50894a0d70 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:47.239010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:47.362997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:47.421050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fef7-b0cf-40c5-9f2d-3a322ae367b3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:47.457851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:47.458814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:47.459097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:47.459405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] PATH_INFO: `/v3/system/users/f94fb4b939e34cbe91be7a50894a0d70/roles/7e5337e63b0b4809b748baf1e1396bf6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:47.460949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f94fb4b939e34cbe91be7a50894a0d70, role_id=7e5337e63b0b4809b748baf1e1396bf6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:47.462377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e20963-0993-4ce4-810f-b5a9bfc8e235 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:47.549984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 170/334] 10.222.0.48 () {58 vars in 1334 bytes} [Fri May 27 09:14:47 2022] PUT /identity/v3/system/users/f94fb4b939e34cbe91be7a50894a0d70/roles/7e5337e63b0b4809b748baf1e1396bf6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:49.982397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57366ec-8a18-4d1f-b663-48357455b480 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:49.982880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57366ec-8a18-4d1f-b663-48357455b480 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:49.983256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57366ec-8a18-4d1f-b663-48357455b480 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:49.985738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 165/335] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:49.993551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae14-5dce-4039-816b-a3bda0b78cc9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:49.994505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae14-5dce-4039-816b-a3bda0b78cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:49.995075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0ae14-5dce-4039-816b-a3bda0b78cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.018124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6cb0ae14-5dce-4039-816b-a3bda0b78cc9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:50.019699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6cb0ae14-5dce-4039-816b-a3bda0b78cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:50.116070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 171/336] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:50.152796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964350-1b67-4953-b753-7558aa64a80b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.153437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964350-1b67-4953-b753-7558aa64a80b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.153906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964350-1b67-4953-b753-7558aa64a80b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.155872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 166/337] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:50.161605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.162190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.162715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.163520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27964350-1b67-4953-b753-7558aa64a80b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:50.184378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:50.185845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:50.430735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:50.467238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 172/338] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:14:50.527773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:50.568932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27964350-1b67-4953-b753-7558aa64a80b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:50.684976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:50.686408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.687165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.687893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.731520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-82159271-8dca-4733-96b7-5420cd582332 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:50.756346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:50.758160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:50.759799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:50.764471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 167/339] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 602 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:50.771350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bd0-2919-4404-a6db-16a146b1f503 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.772151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bd0-2919-4404-a6db-16a146b1f503 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.772854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14bd0-2919-4404-a6db-16a146b1f503 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.776027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 168/340] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:50.780850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14bd0-2919-4404-a6db-16a146b1f503 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:50.799762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9c0-7a0b-4818-a708-6c369afce85c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:50.806058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:50.882014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:50.883389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.883881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.884376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:50.927275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-564f8713-d87e-49b1-9013-92976eaec537 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:50.954661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:50.956409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:50.958012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 173/341] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:50.959390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:50.960390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:50.960390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:51.007098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:51.009206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:51.036566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14bd0-2919-4404-a6db-16a146b1f503 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:51.078657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:51.079340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:51.079789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:51.080243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:51.081185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:51.081649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:51.083300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:51.093921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 169/342] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:51.095151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:51.165875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:51.181964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:51.225379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 174/343] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:51.226569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:51.233460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679507c4-a82b-460b-a6ad-65858b4dba44 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:51.342434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:51.343364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:51.343786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:51.344194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:51.345094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:51.345579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:51.347165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:51.371597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 170/344] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:51.372744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:51.462119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc3824-2469-4519-8dd1-4c0ea5e0fa01 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:51.529483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7c03-78a0-4bd1-9061-415f3563493b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:51.546228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:51.547134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:51.547537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:51.547930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:51.590878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:51.611631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:51.613444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:51.613444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:51.613444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:51.636782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:51.645678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:51.646788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:51.647684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:51.656808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 171/345] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:51.664225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:51.686755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:51.727713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:51.729214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:51.879017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b82a1-1375-4fca-bb31-2fc3d97b96da None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:51.955456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:51.956578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:51.957074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:51.957568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.056834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 175/346] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 830 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:52.060425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:52.125081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:52.151659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:52.155208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b744a5-deb4-4afe-9d65-709853093ded None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:52.197980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:52.227716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f05f-555b-4994-a8d5-0cc15b11d113 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:52.288793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:52.289652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.290051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.290520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.291462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:52.291952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:52.293878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:52.309115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 176/347] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:52.310616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:52.334683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 172/348] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:52.336558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e89ee-3293-4906-810a-2268ae55e8c1 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.336977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e89ee-3293-4906-810a-2268ae55e8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.337386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e89ee-3293-4906-810a-2268ae55e8c1 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.339279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 173/349] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:52.341635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848d72-b7af-4248-9f62-e270ac12ba71 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.342417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848d72-b7af-4248-9f62-e270ac12ba71 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.343151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848d72-b7af-4248-9f62-e270ac12ba71 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.345891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 174/350] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:52.347627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848d72-b7af-4248-9f62-e270ac12ba71 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:52.470901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636a25-2595-45cf-8a70-92725e298afb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:52.520955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:52.524974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:52.525853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.526137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.526462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.527773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:52.529279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:52.532942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 177/351] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:52 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:52.532942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:52.544893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848d72-b7af-4248-9f62-e270ac12ba71 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:52.552503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:52.601241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:52.601905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.602176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.602678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.708269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe905-c179-4876-be90-0b3209e5ea3b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:52.736925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:52.738177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:52.760942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:52.764458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.764458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.764458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.764458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:52.765781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8db47-8d14-4f8c-9e99-ea686ec78da8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:52.767943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 178/352] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:52 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:52.769367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a922b-4c56-4a6a-8f7e-21242b6910e2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.769805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a922b-4c56-4a6a-8f7e-21242b6910e2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.770272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a922b-4c56-4a6a-8f7e-21242b6910e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.796272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ef7a922b-4c56-4a6a-8f7e-21242b6910e2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:52.798806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ef7a922b-4c56-4a6a-8f7e-21242b6910e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:52.879977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:52.888641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 179/353] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:52.890357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:52.890791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:52.899086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:52.899299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 175/354] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:52.899299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:52.909499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:52.911067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:52.923121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.002755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 180/355] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:53.003947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.049335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ea772-5b98-4979-88bb-b4a522cf379e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.111238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.112136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.112404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.112839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.122297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:53.124578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cc4b7-f896-45ce-9710-56149fd2a16c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:53.127986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 176/356] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:52 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:53.131196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dc7ef-c195-41d6-b6ae-ec1b55961c01 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.131633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dc7ef-c195-41d6-b6ae-ec1b55961c01 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.131963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dc7ef-c195-41d6-b6ae-ec1b55961c01 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.138251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 177/357] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:53.138251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dc7ef-c195-41d6-b6ae-ec1b55961c01 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.161005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a5a5a-4078-4605-9a66-ded9c4d82b4f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.206036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.207948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.207948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.207948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.209665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:53.211196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:53.213238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 181/358] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:53.214085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.232628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.238800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.315086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dc7ef-c195-41d6-b6ae-ec1b55961c01 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.350985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d34b85-b188-4fbf-96bf-6833efdb84be None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.373654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.375376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.375795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.376541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.378827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:53.381462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:53.383863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 178/359] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:53.385598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.415993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.417380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.418295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.419029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.438390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:53.449669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:53.451369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 182/360] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:53.452933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.566843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db9007-8fce-4a1a-bb6c-0db0ca83a5b4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.579622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.610137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.630115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.631710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28c6ce-17a6-48fa-bfd2-817cb8b9edde None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.632171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.632435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.632894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.652624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b8d42459-19fc-42e7-b950-ecd673d49bc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:53.663116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:53.664766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:53.666149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 179/361] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:53.667408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.667617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.668297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.686714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.687787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.688252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.688704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.689811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=06620fd9d314432ab7107312bc93402f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:53.690331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:53.692528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:53.693225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:53.696354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:53.704372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 183/362] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:53.705589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.793914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 180/363] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:53.793914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.906690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d21fa5-465b-4621-87ea-14d292a4aae3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:53.962423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:53.964243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:53.964704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:53.965227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:53.966618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0d776cda9a8749df88d778d848b78e4e) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:53.967139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:53.969556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:53.972770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:53.983111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 184/364] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:53.983111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:53.995752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabf5de-71cd-4fa3-a313-238175de455a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.033376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:54.073642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.075477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.076160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.076814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:54.079270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:54.082342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:54.086005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 181/365] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:54.087779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:54.218846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345c7b-49c7-4a80-ac7e-64d14932b47c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.282833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.283943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.284624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.285552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:54.287393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:54.288144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:54.291136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:54.306575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 185/366] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:54.308950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:54.315004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8e28e-21f0-4527-9d05-f0fa50362d21 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.391318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.393611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.394631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.395599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:54.397561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:54.398336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:54.401546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:54.421862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 182/367] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:54.423984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:54.450593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:54.515582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:54.559198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06fea-897a-4360-9cbc-7dba920c0f11 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.626811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230d773-1975-49ba-afbd-e7ed58a5f2f7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.628418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.629686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.630326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.630788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:54.650236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:54.666067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:54.667095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 09:14:54.667955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:54.668575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 186/368] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:54.669751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:54.691682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.692892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.693558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.694106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:54.695975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=06620fd9d314432ab7107312bc93402f) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:54.696571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:54.699536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:54.710287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 183/369] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:14:54 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:54.712173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:54.890746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8239-bbcf-4323-8b23-c4ac398619db None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.939768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee93e0-7cb4-4397-9468-1ebde1702ea8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:54.957954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:54.968116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:54.996832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:54.998475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:54.999254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:54.999998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:55.056225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:55.057810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:55.059272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:55.061285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:55.170324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:55.173513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:55.221793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:55.224808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:55.394881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 187/370] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:55.397906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:55.478701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:55.614162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 184/371] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:14:55.616945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:55.634078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:55.670574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5657a95-ef15-48ce-8aa8-a9f3b822dd76 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:55.715982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:55.717108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:55.717604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:55.718116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:55.720061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:55.720852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:55.725032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:55.746731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 188/372] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:55.748043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:55.855878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:55.876145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4d89a-026f-4713-ab63-770e2aa322a1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:55.904955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1b6b5-7af4-449b-bcba-1e129e85f581 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:55.966119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:55.967017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:55.967299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:55.967573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:55.970545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:55.972032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:55.973812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 189/373] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:55 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:55.974643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:55.994810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:55.995700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:55.996016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:55.996301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] PATH_INFO: `/v3/projects/822861f091c54d5980ee4872d75a26fa` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.000206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] RBAC: Authorizing `identity:get_project(project_id=822861f091c54d5980ee4872d75a26fa)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.001754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.003666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 185/374] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:55 2022] GET /identity/v3/projects/822861f091c54d5980ee4872d75a26fa => generated 344 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.004482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:56.152937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:56.183051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fefaaf-8309-4cc1-a7ae-42863282ff24 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.210862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aac0d6-f5d4-4afe-a367-804dc265475c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.259183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.260041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.260317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.260584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] PATH_INFO: `/v3/users/f94fb4b939e34cbe91be7a50894a0d70` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.262614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] RBAC: Authorizing `identity:get_user(user_id=f94fb4b939e34cbe91be7a50894a0d70)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.264307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.266587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 190/375] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:14:55 2022] GET /identity/v3/users/f94fb4b939e34cbe91be7a50894a0d70 => generated 293 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.267571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:56.285966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.287127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.287589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.288012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.289623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.291566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.293864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:56.295426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 186/376] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.296435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:56.408869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de33e8-1fd8-4e69-a486-a3c8735b325c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.410183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385810c-cd7c-4aaa-8a41-d4996cd1a360 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.450964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.451888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.456496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.456496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.456496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] RBAC: Adding query filter params (role.id=7e5337e63b0b4809b748baf1e1396bf6, user.id=f94fb4b939e34cbe91be7a50894a0d70, scope.system=all) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:56.456496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.456496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ad284-911a-444d-a9da-f9a0bd5f3c5e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.466893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:56.470629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 187/377] 10.222.0.48 () {56 vars in 1308 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/role_assignments?user.id=f94fb4b939e34cbe91be7a50894a0d70&scope.system=all&role.id=7e5337e63b0b4809b748baf1e1396bf6 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.472807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f041-d87a-4cec-8a2e-d36e7dcae674 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.473328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f041-d87a-4cec-8a2e-d36e7dcae674 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.473780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9f041-d87a-4cec-8a2e-d36e7dcae674 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.479954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 188/378] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:56.481745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9f041-d87a-4cec-8a2e-d36e7dcae674 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:56.487975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.489513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.490008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.490535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] PATH_INFO: `/v3/groups/89766b3629ae4749a681e8e88aac739c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.513904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] RBAC: Authorizing `identity:get_group(group_id=89766b3629ae4749a681e8e88aac739c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.516493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaaa168-23bd-4d7c-9736-b0f4460dfbd1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.526303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 191/379] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/groups/89766b3629ae4749a681e8e88aac739c => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.528851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.529135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.529605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.588014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:56.590752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:56.615762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9f041-d87a-4cec-8a2e-d36e7dcae674 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.657136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.658275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.658793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.659277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.660375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=0d776cda9a8749df88d778d848b78e4e) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:56.660930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.662714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.669567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 189/380] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.670813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:56.758828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:56.814121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd54666-7639-4673-bd1c-24710ed5d043 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:56.856836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:56.857952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:56.858426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:56.858854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:56.859886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] RBAC: Adding query filter params (group.id=89766b3629ae4749a681e8e88aac739c, role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=822861f091c54d5980ee4872d75a26fa) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:56.860387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:56.862075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:56.869151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 190/381] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/role_assignments?group.id=89766b3629ae4749a681e8e88aac739c&scope.project.id=822861f091c54d5980ee4872d75a26fa&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:56.897243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 192/382] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:56.928752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:57.072749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f2b-d414-45c6-806b-5f2f5ddd235b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:57.114375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:57.128676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:57.129537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:57.129823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:57.133535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:57.149182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:57.157722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:57.159449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:14:57.160759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:57.165274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 191/383] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:14:57.171961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:57.213191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:57.416009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea05b6-0c6c-42a4-a4ad-ffa8f6094dc7 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:57.501869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:57.503266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:57.503813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:57.504302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:57.505814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:57.506387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:57.509025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:57.527869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 193/384] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:14:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:14:57.533738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:57.805350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606a299-27da-4ae8-84eb-445fcaec18c0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:57.860542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:57.861421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:57.861692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:57.861961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:57.863354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:57.864848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:57.866991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 192/385] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:57 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:57.875387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:57.973738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:58.011183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b201-e755-449c-b160-75ab5ce59e46 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:58.055060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:58.056015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:58.056307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:58.056548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:58.057755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:58.059380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401b3e7-9d86-4582-952c-f3ed6ef89feb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:58.061526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 194/386] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:57 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:58.067127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:58.148220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:58.210878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854aac9-07f5-4ea9-9778-12d48cccd536 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:58.262639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:58.263537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:58.263815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:58.264072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/groups/9ab0435c1e7a453e986b6051d8700131/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:58.272399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, group_id=9ab0435c1e7a453e986b6051d8700131, role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:58.273880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1490270-88b7-4909-b656-51bb1bdd4b3c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:58.356845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 193/387] 10.222.0.48 () {58 vars in 1442 bytes} [Fri May 27 09:14:58 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/groups/9ab0435c1e7a453e986b6051d8700131/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:14:58.439841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467f53-0272-41b5-b116-9b98d88d4d6a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:58.441362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467f53-0272-41b5-b116-9b98d88d4d6a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:58.442107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467f53-0272-41b5-b116-9b98d88d4d6a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:58.445332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 195/388] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:14:58.455488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:58.456487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:58.457270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:58.499131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:58.501793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:58.642399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 194/389] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:58.680225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467f53-0272-41b5-b116-9b98d88d4d6a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:58.899875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467f53-0272-41b5-b116-9b98d88d4d6a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:58.965268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:58.966966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:58.967650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:58.968353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:58.989132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-eb1e40fc-40c7-427c-9f93-f67dbfd764f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:59.005984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:59.008038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:59.009786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:14:59.015110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 196/390] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:59.021647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:59.087566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:59.100266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40425936-12ae-4624-ae85-bfa7a47fcc14 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:59.100532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40425936-12ae-4624-ae85-bfa7a47fcc14 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:59.100810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40425936-12ae-4624-ae85-bfa7a47fcc14 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:59.102287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 197/391] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:14:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:14:59.111013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:59.111291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:59.111566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:59.121314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:59.144910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:14:59.146430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:14:59.226034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be34d7-9b25-4071-ba4c-2436e0688a2d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:59.311500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 198/392] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:14:59.314531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:59.315906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:59.316606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:59.317262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:59.318887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:14:59.319598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:59.322786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:59.354689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 195/393] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:59.364579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:59.377058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:59.616886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872b584-2c65-4394-b901-65c8e80da068 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:59.686468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b2eb5-e90f-4ab9-9467-432ee3575b68 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:14:59.709574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:59.711814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:14:59.728710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:59.730318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:59.730782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:59.731257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:59.740423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:14:59.743172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:14:59.751970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 199/394] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:14:59 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:14:59.756253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:14:59.788380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:14:59.789731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:14:59.790268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:14:59.790771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:14:59.827938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:14:59.858232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:14:59.859909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:14:59.861385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:14:59.866167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 196/395] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:14:59.876213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:00.030741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d820-9b95-400d-b27f-7a361eb36bec None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:00.130099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:00.130898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.131184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.131461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.134095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:00.135602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:00.137887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 200/396] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:14:59 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:00.145353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77446ef4-b455-42d9-beb1-f7b445a95500 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:00.146377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:00.224516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:00.225193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.225479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.225752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.226556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:00.226867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:00.228269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:00.250734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 197/397] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:00.259070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:00.299870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:00.314826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:00.351865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8abd3-f5d0-49b8-9689-be5fc794a860 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:00.421386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:00.422936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.423436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.423927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/groups/9ab0435c1e7a453e986b6051d8700131/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.427838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, group_id=9ab0435c1e7a453e986b6051d8700131, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:00.430519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9576e63-15b2-419f-b7fd-e930b4cf534d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:00.452956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8622-9520-47c9-9895-f509842ef653 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:00.522396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 201/398] 10.222.0.48 () {58 vars in 1442 bytes} [Fri May 27 09:15:00 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/groups/9ab0435c1e7a453e986b6051d8700131/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:15:00.522832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:00.523456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.523734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.524022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.525464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:00.525857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1bf1-c7e7-4208-98c2-5ea45fcf97ef None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.534678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1bf1-c7e7-4208-98c2-5ea45fcf97ef None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.534678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c1bf1-c7e7-4208-98c2-5ea45fcf97ef None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.535071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bebc75-a15e-4950-9fb4-e879455e50d8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:00.535071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 198/399] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:00 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:00.535303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 202/400] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:00.535428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844f96-5e2d-4a25-9f64-2d3d845376e5 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.535428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844f96-5e2d-4a25-9f64-2d3d845376e5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.535428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844f96-5e2d-4a25-9f64-2d3d845376e5 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.535428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 199/401] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:00.539538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1bf1-c7e7-4208-98c2-5ea45fcf97ef None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:00.540268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c29a4-3f06-4a84-a739-e3084dc0fd96 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.540526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c29a4-3f06-4a84-a739-e3084dc0fd96 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.540801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c29a4-3f06-4a84-a739-e3084dc0fd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.563974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8e3c29a4-3f06-4a84-a739-e3084dc0fd96 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:00.565304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8e3c29a4-3f06-4a84-a739-e3084dc0fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:00.669556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 200/402] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:00.669556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.669556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.669556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.705108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:00.707558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:00.768737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:00.780589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c1bf1-c7e7-4208-98c2-5ea45fcf97ef None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:00.831174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 201/403] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:00.832369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:00.839736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:00.854738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:00.857781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:00.858714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:00.859437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:00.864226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:00.865425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:00.867198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 203/404] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:00 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:00.868113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.013589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848dcd7-f2a4-4d87-91e6-99612533a63c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.047180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb761dc-2dc5-4058-9cf4-4f51b9b5b398 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.059356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.060181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.060463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.060737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.075655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:01.085880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:01.087642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 202/405] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:01.089214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.123279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.124951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.125595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.126257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.138649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:01.145937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:01.161546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:01.163571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:01.165384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:01.169907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 204/406] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:01.171799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.282777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:01.284300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64963-8e66-4810-9b28-b674b3ebbcc1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.344793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.346762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.347507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.348315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f/groups/9ab0435c1e7a453e986b6051d8700131/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.352009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=06620fd9d314432ab7107312bc93402f, group_id=9ab0435c1e7a453e986b6051d8700131, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:01.355125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:01.372204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc7f2-ae89-4133-95e0-14394422838a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.416904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.417807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.418780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.419197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.420260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:01.420686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:01.422238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:01.438352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 205/407] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:01.438352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.483535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 203/408] 10.222.0.48 () {58 vars in 1442 bytes} [Fri May 27 09:15:01 2022] PUT /identity/v3/projects/06620fd9d314432ab7107312bc93402f/groups/9ab0435c1e7a453e986b6051d8700131/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:01.485521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.714459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:01.759992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857a864-55f0-48e5-a068-4e49f0204c18 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.775025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4a6f-b8fe-463e-b161-39a95418b5c6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:01.827944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:01.867528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.869118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.869874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.872248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.873502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:01.874174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:01.875183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:01.875370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:01.875651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:01.876107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:01.878450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:01.879076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:01.881872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:01.890249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 204/409] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:01 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:01.894742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:01.905164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 206/410] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:01.914372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:02.116611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34ebe-5e4a-4b8a-b698-34e8a3e2ea9d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:02.120050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd7092-95c0-4b5e-b77b-8dbe00dc0c12 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:02.179807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:02.181329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.181788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.182427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.184886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:02.186805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:02.190137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 205/411] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:01 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:02.196921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:02.218993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:02.243433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:02.245711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.245711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.245711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] PATH_INFO: `/v3/groups/89766b3629ae4749a681e8e88aac739c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.251802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] RBAC: Authorizing `identity:get_group(group_id=89766b3629ae4749a681e8e88aac739c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:02.254619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:02.257628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 207/412] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:01 2022] GET /identity/v3/groups/89766b3629ae4749a681e8e88aac739c => generated 229 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:02.267931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:02.329148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1d2f-a330-4cdf-973d-3c7e442eb0c7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:02.371953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87460f64-3311-474f-995f-7f998420ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:02.373098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87460f64-3311-474f-995f-7f998420ffff None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.373699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87460f64-3311-474f-995f-7f998420ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.374113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87460f64-3311-474f-995f-7f998420ffff None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e/groups/9ab0435c1e7a453e986b6051d8700131/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.382261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87460f64-3311-474f-995f-7f998420ffff None admin] RBAC: Authorizing `identity:create_grant(project_id=0d776cda9a8749df88d778d848b78e4e, group_id=9ab0435c1e7a453e986b6051d8700131, role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:02.385600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87460f64-3311-474f-995f-7f998420ffff None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:02.414734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:02.472341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ab166-2681-42da-b02b-e09fbd5f0593 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:02.478704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 206/413] 10.222.0.48 () {58 vars in 1442 bytes} [Fri May 27 09:15:02 2022] PUT /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e/groups/9ab0435c1e7a453e986b6051d8700131/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:02.535275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:02.536513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.536951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.537378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] PATH_INFO: `/v3/projects/822861f091c54d5980ee4872d75a26fa` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.539010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] RBAC: Authorizing `identity:get_project(project_id=822861f091c54d5980ee4872d75a26fa)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:02.540686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3301dfb-2360-42c6-8930-abc16a8e10d4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:02.543828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 208/414] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:02 2022] GET /identity/v3/projects/822861f091c54d5980ee4872d75a26fa => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:02.560426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87460f64-3311-474f-995f-7f998420ffff None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:02.611746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae8bf2-75b0-45c8-9e7e-9aa1b4ff8ebf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.612263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae8bf2-75b0-45c8-9e7e-9aa1b4ff8ebf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.612678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae8bf2-75b0-45c8-9e7e-9aa1b4ff8ebf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.614799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 209/415] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:02.623721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.624121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.624620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.687716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:02.696121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:02.772313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:02.825887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87460f64-3311-474f-995f-7f998420ffff None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:02.846408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 210/416] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:02.888981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:02.890179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:02.891750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:02.891872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:02.892319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] PATH_INFO: `/v3/projects/822861f091c54d5980ee4872d75a26fa/groups/89766b3629ae4749a681e8e88aac739c/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:02.896191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] RBAC: Authorizing `identity:create_grant(project_id=822861f091c54d5980ee4872d75a26fa, group_id=89766b3629ae4749a681e8e88aac739c, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:02.898951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:02.950975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:02.998229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 207/417] 10.222.0.48 () {58 vars in 1442 bytes} [Fri May 27 09:15:02 2022] PUT /identity/v3/projects/822861f091c54d5980ee4872d75a26fa/groups/89766b3629ae4749a681e8e88aac739c/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:03.016584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef965-da05-42a8-9536-0e28ce93bbb6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:03.059931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:03.061143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:03.061610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:03.062052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:03.075399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:03.084272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:03.085238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:03.086044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:03.088603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 211/418] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:15:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:03.098090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:03.363516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5fd5a-4a16-4e4b-99c7-39c75c1f3e63 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:03.406928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:03.412393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:03.421667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:03.429581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 208/419] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:03.438437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:03.562167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcd0d1-115c-4549-9857-0e534d03d0ab None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:03.571564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:03.602907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:03.604109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:03.604865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:03.605145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:03.686796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:03.688447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:03.832728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 212/420] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 09:15:03.840879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:04.118721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000889bd-d526-4504-b1e6-476be8944188 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:04.210245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:04.211854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:04.212350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:04.212816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:04.215629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:04.218515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e88a-952c-45bd-aba5-96512780ada1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:04.222154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 209/421] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:03 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:04.231610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:04.258718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:04.336706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ec34a-d43c-4203-b658-4c978417e5a4 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:04.337187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ec34a-d43c-4203-b658-4c978417e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:04.337560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ec34a-d43c-4203-b658-4c978417e5a4 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:04.341308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 210/422] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:04.360319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:04.360761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:04.361227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:04.406103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:04.406103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:04.462354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de203-fc77-475d-9906-9a163cb4ccf2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:04.552222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:04.554210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:04.554959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:04.555701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:04.563512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:04.566163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:04.579117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:04.584053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 213/423] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:04 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:04.592469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:04.594430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 211/424] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:04.669549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:04.743695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:04.795623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910263-29d5-442f-bd95-91d844a7cfac None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:04.917394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc440b-adcc-4966-80bc-bc407d01b948 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:04.920775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:04.921804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:04.922495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:04.922769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:04.924510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=06620fd9d314432ab7107312bc93402f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:04.925230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:04.934233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:04.954873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 214/425] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:15:04 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 611 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:05.006073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:05.007659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:05.008130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:05.008606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:05.040453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:05.062434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:05.064167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:15:05.065801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:05.070491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 212/426] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:05.081374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:05.276495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e07e-3de8-47c5-9a19-cbb235b8f0f4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:05.342232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:05.342232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:05.342232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:05.342232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:05.342892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:05.342892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:05.342892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:05.360268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 215/427] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:05.362188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:05.369656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:05.468862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:05.535609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40272cb-b45f-488e-bf6f-b5c8aee8d416 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:05.586853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:05.587525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:05.587811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:05.588087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:05.700393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:05.702603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b726ac-361d-46a5-bf54-fe50d220e14d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:05.811457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:05.904923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 213/428] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:05.914080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:05.972268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ce6f-954f-403e-8469-a55cf5daa464 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:05.972760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ce6f-954f-403e-8469-a55cf5daa464 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:05.973596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ce6f-954f-403e-8469-a55cf5daa464 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:05.976956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 214/429] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:15:05.982313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:05.982599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:05.982877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.004360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:06.006338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:06.086096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0942a-98ca-4ade-81b8-7bcafa42c9ef None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.138952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 215/430] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:06.162563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.164091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.164582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.165071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.167652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:06.170663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:06.179665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.181643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 216/431] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:05 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:06.184211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.319776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.342745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.366869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c24982-b981-466c-b51b-6b36c675d117 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.369458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee5252-ea0d-465c-9bc4-4e868665d659 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.411887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.412759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.413029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.413319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.414588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:06.415431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.416050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deaaa0-91af-43c5-94fe-d37e1621f60d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:06.416343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.416664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.416974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.417749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 217/432] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:06.419650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb221-9ec4-49f3-bf3b-0bc74c67e5bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.419928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb221-9ec4-49f3-bf3b-0bc74c67e5bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.420200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdb221-9ec4-49f3-bf3b-0bc74c67e5bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.421467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 218/433] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:06.422599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075daf-f8ba-4ed5-8e33-60f53289d9da None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.422876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075daf-f8ba-4ed5-8e33-60f53289d9da None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.423146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075daf-f8ba-4ed5-8e33-60f53289d9da None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.424403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 219/434] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:06.425123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4075daf-f8ba-4ed5-8e33-60f53289d9da None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.443908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a0d33099-829a-44a8-be0a-2368669dbdd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:06.456638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:06.458105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 216/435] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:06.460505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0006f-ec02-455c-a540-f4cbd3dd0a9e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.462169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0006f-ec02-455c-a540-f4cbd3dd0a9e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.462169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0006f-ec02-455c-a540-f4cbd3dd0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.478900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1fd0006f-ec02-455c-a540-f4cbd3dd0a9e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:06.480105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1fd0006f-ec02-455c-a540-f4cbd3dd0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:06.550654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4075daf-f8ba-4ed5-8e33-60f53289d9da None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.558760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 217/436] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:06.560828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.561276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.561709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.584278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:06.585487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:06.604880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.606023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.606466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.606762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.607596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=0d776cda9a8749df88d778d848b78e4e) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:06.607903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:06.609379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:06.612652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.616382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 220/437] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:06.617227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.670280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 218/438] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:06.675738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.684320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.795133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f7ce-47a8-44a5-9153-0197a443eb4b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.852597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223ba3-3894-4791-9f8f-4f5d1ae4f6ed None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:06.853934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.854777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.855797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.856187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.857208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:06.857651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:06.859577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:06.872461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 221/439] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:06.873894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.899475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:06.900622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:06.900886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:06.901155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:06.912380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:06.920726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:06.922417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 219/440] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:06.924012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:06.950495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.994334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:06.995647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc26055-b528-4c9b-b813-002d86dd61dd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.033767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.034593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.034867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.035133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.036063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59dbd-cae9-414f-ad45-8836b8bbf30b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.048508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:07.056831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:07.059219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 222/441] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:07.061266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.090831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.091676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.092041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.092461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.175992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.178544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.179125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270221-3830-4b08-85fd-d2850d7bb894 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.214164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.232045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.233177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.233635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.234075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.235362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:07.235869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.238303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.251224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 223/442] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:07.252632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.278239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.327345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 220/443] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:07.328282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.390882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc3118-be89-475a-97c7-a5cd9c67a4c7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.428961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.429545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.429834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.430110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.430988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:07.431295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.432703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.441570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 224/444] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:07.442524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.493956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.496110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1989c9-dcd2-4464-abee-bb69e6e5b262 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.554706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.555836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.556341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.556855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.562648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f767c-bb62-43b8-a458-18e64799808b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.598029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.612690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.614121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.614653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.615203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] PATH_INFO: `/v3/projects/822861f091c54d5980ee4872d75a26fa` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.617420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] RBAC: Authorizing `identity:get_project(project_id=822861f091c54d5980ee4872d75a26fa)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.620224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.629564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 225/445] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/projects/822861f091c54d5980ee4872d75a26fa => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:07.629564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.640387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.641900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.770096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1229d9-65c2-4134-b349-6627310ed75e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.786674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.790525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 221/446] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:07.791416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.830318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.832747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.833251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.833726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.865142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:07.910801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a101de3-e9f6-4e2d-a0e4-a3f0d070d7f7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:07.951149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:07.952744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:07.953203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:07.953604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] PATH_INFO: `/v3/groups/89766b3629ae4749a681e8e88aac739c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:07.955133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] RBAC: Authorizing `identity:get_group(group_id=89766b3629ae4749a681e8e88aac739c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.956785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:07.958620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 222/447] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/groups/89766b3629ae4749a681e8e88aac739c => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:07.959470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:07.959630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:07.962263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.102970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 226/448] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:08.103726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.116367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71652573-4602-4c7a-ac3f-a1621786331f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.122117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:08.160795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:08.167112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.168343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.168683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.168986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] PATH_INFO: `/v3/projects/06620fd9d314432ab7107312bc93402f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.170539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] RBAC: Authorizing `identity:get_project(project_id=06620fd9d314432ab7107312bc93402f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.172669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.174740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 223/449] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:07 2022] GET /identity/v3/projects/06620fd9d314432ab7107312bc93402f => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:08.175703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.242398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0877-109e-458f-a1af-c33fc3ab29d9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.285274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.286394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.286850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.287350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.288946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] RBAC: Adding query filter params (group.id=89766b3629ae4749a681e8e88aac739c, role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=822861f091c54d5980ee4872d75a26fa) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:08.289524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.292033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.303177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 227/450] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/role_assignments?group.id=89766b3629ae4749a681e8e88aac739c&scope.project.id=822861f091c54d5980ee4872d75a26fa&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:08.303873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed9f84-b35a-4fb7-881a-c546e56fabba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.304529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.370602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.371983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.372426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.372864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] PATH_INFO: `/v3/projects/0d776cda9a8749df88d778d848b78e4e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.375176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] RBAC: Authorizing `identity:get_project(project_id=0d776cda9a8749df88d778d848b78e4e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.377571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.380867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 224/451] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/projects/0d776cda9a8749df88d778d848b78e4e => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:08.389992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.454059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:08.498436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c24f-a87c-4fd0-abec-c3d6577af2b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.509544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:08.548606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.549942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.551276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.551276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.553496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.555049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.557834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 228/452] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:08.567861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.587897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e790-1ab0-4241-b7bf-bf46fe68dc77 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.644077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.645029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.645392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.645745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] PATH_INFO: `/v3/groups/9ab0435c1e7a453e986b6051d8700131` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.647233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] RBAC: Authorizing `identity:get_group(group_id=9ab0435c1e7a453e986b6051d8700131)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.648868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.650867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 225/453] 10.222.0.48 () {56 vars in 1180 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/groups/9ab0435c1e7a453e986b6051d8700131 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:08.660453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:08.702047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823c6bb-2e18-4910-a676-9b47b0c183c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.739234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.739881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.740171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.740445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.741293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=8ff134ce9e0f40f38af26ff674f3b83f, scope.project.id=06620fd9d314432ab7107312bc93402f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:08.741604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.743143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adbc7-7e3b-47d2-a5be-0145a0bc1a14 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.750703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 229/454] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=06620fd9d314432ab7107312bc93402f&role.id=8ff134ce9e0f40f38af26ff674f3b83f => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:08.819641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346064-16b9-4861-ad9e-6041525daf59 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:08.878622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:08.879498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:08.879776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:08.880047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:08.880900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] RBAC: Adding query filter params (group.id=9ab0435c1e7a453e986b6051d8700131, role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=0d776cda9a8749df88d778d848b78e4e) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:08.881207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:08.882770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b076ac-5942-4eac-a60b-6cf5b0cad9a8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:08.886359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:08.902647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 226/455] 10.222.0.48 () {56 vars in 1376 bytes} [Fri May 27 09:15:08 2022] GET /identity/v3/role_assignments?group.id=9ab0435c1e7a453e986b6051d8700131&scope.project.id=0d776cda9a8749df88d778d848b78e4e&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:12.212324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b692-a323-4d48-95b0-1c1388389950 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.212850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b692-a323-4d48-95b0-1c1388389950 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.212850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b692-a323-4d48-95b0-1c1388389950 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.214639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 230/456] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:12.220547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce8be-fce5-42e2-804e-3e195b6d414a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.221028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce8be-fce5-42e2-804e-3e195b6d414a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.221101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ce8be-fce5-42e2-804e-3e195b6d414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.238830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b51ce8be-fce5-42e2-804e-3e195b6d414a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:12.240018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b51ce8be-fce5-42e2-804e-3e195b6d414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:12.316661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 227/457] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:12.342609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45b692-a323-4d48-95b0-1c1388389950 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:12.357317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:12.483192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45b692-a323-4d48-95b0-1c1388389950 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:12.535971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:12.536871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.537142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.537410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.560377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:12.562001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:12.594802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:12.595786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:12.596662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:12.596662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:12.596662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:12.596662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:12.596662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 231/458] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:12.600900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375c09-e221-4ea2-88ed-d2fa6857452c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.601208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375c09-e221-4ea2-88ed-d2fa6857452c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.601453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375c09-e221-4ea2-88ed-d2fa6857452c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.602850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 228/459] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:12.605649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:12.608854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0914-2022-4144-895b-c14be4a66245 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.609135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0914-2022-4144-895b-c14be4a66245 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.609411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0914-2022-4144-895b-c14be4a66245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.635444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c49e0914-2022-4144-895b-c14be4a66245 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:12.637225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c49e0914-2022-4144-895b-c14be4a66245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:12.703005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:12.718471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 229/460] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:12.720189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef91e5-d0d6-4ef3-91f6-1415d6ce7b89 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.720673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef91e5-d0d6-4ef3-91f6-1415d6ce7b89 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.721165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef91e5-d0d6-4ef3-91f6-1415d6ce7b89 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.722841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 230/461] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:12.728886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.729512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.729981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.754262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:12.756075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:12.759479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579048-6c9d-4939-a86b-160c2f71e4b1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:12.805270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:12.806353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:12.806877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:12.807395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:12.808381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:12.809498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:12.811263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:12.824415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:12.827114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 232/462] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:12.828184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:12.849032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 231/463] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:12.851774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:12.973383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b573a-69aa-4fd2-80df-a8106d91124c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:12.987378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d89bfd-c7d8-4251-8e00-8200c45cab9c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:12.999932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.014676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.015397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.015676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.015945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.026912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.028444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.028588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.029112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.029391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.029734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.030426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.032073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c54d25-dcd3-44e5-b99d-1d66cb1f69bf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:13.048688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:13.051705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 233/464] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:13.052564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.129431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 232/465] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:12 2022] POST /identity/v3/users => generated 246 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:13.130810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e635-94b6-4fb5-901f-b64304f9f3f3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.131136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e635-94b6-4fb5-901f-b64304f9f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.131417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e635-94b6-4fb5-901f-b64304f9f3f3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.132950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 233/466] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:13.138696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76e635-94b6-4fb5-901f-b64304f9f3f3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.188863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b53d08-f736-4b9c-b694-b8ee3aff6a1c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.201831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.244833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.247247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.247534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.247808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.266705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.269442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.271922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76e635-94b6-4fb5-901f-b64304f9f3f3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-dd637f9b-9e9f-489c-8c38-c7379a7e4981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:13.294552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:13.295509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:13.296355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:13.296355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:13.296355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.296355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:13.299976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 234/467] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:13.302504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10714-6305-4e75-8271-60f154419180 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.303284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10714-6305-4e75-8271-60f154419180 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.303755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10714-6305-4e75-8271-60f154419180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.315424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.316113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.316394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.316666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.317427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:13.317745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.319211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.329930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b5c10714-6305-4e75-8271-60f154419180 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:13.332555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b5c10714-6305-4e75-8271-60f154419180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:13.335580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 234/468] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:13.336558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.363003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.429771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 235/469] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:13.432817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c10714-6305-4e75-8271-60f154419180 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.506645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db93a7a-7a15-4ec8-bfbe-2dca7b335ebd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.516532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.550592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.551172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.551482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.551827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.552665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:13.552974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.554386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.572282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 235/470] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:13.573342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.609583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c10714-6305-4e75-8271-60f154419180 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.650332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.651013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.651341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.651625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.653172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.655284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.695048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a485f-f36a-4334-b6bc-db38a3c3ad49 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.731220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.732007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.732285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.732560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.743742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 236/471] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:13 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:13.745658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.750488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.752049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:13.763736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9c970648-5c4c-4556-b586-bb5f82385c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:13.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:13.775448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:13.776497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:13.776497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:13.776497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:13.776497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:13.776497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 236/472] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:13.778651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4712b50-2c03-4ced-aff7-538cde81610b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.778651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4712b50-2c03-4ced-aff7-538cde81610b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.778651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4712b50-2c03-4ced-aff7-538cde81610b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.784229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 237/473] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:13.787673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.787673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.787673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.808510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:13.810168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:13.827714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:13.878372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c945e83-099d-43e2-a677-532514d1f888 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:13.891339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 238/474] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:13.892555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:13.947805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:13.948488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:13.948766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:13.949613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:13.950365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:13.951852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:14.016029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd81d-7724-4a22-beda-9f2e6a034ef3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:14.058716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:14.059484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:14.059763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:14.060033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:14.060762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:14.061065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:14.062557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:14.075554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 239/475] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:14.076242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:14.090945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 237/476] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:13 2022] POST /identity/v3/users => generated 248 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:14.099045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:14.102578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:14.216784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f6f0-1fca-43fa-90f6-3bab2171b88e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:14.253555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:14.266255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:14.266947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:14.267224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:14.267492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:14.274401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:14.274401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:14.302304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41658eb-cd4c-4c93-8743-b34b6310e47a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:14.336078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.sql.core [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 09:15:14.336078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 09:15:14.336078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [parameters: {'id': 'a4005bfe64894173b66e2ef5108a9e15', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 09:15:14.336078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 09:15:14.358514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:14.359210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:14.359501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:14.359766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:14.360489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() May 27 09:15:14.360953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return ref.to_dict() May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:15:14.361849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application yield resource May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.rollback() May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.force_reraise() May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise self.value May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application session.commit() May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 09:15:14.362798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._prepare_impl() May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.flush() May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._flush(objects) May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application compat.raise_( May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application flush_context.execute() May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rec.execute(self) May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 09:15:14.364224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:15:14.365119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:15:14.365937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [parameters: {'id': 'a4005bfe64894173b66e2ef5108a9e15', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:14.366767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 09:15:14.368910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.369991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:14.370073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 240/477] 10.222.0.48 () {60 vars in 1131 bytes} [Fri May 27 09:15:14 2022] POST /identity/v3/roles => generated 144 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:14.370073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:14.455404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:14.477498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 238/478] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:14 2022] POST /identity/v3/users => generated 249 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:14.504003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ace89-e91c-4360-b946-3ed72a8a6db6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:14.551912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:14.553166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:14.553663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:14.554152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:14.567471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-304e6ce3-ba56-404f-b571-adf968975442 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:14.577846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:14.578862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 09:15:14.579647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:14.582604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 241/479] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:15:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:14.588558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:14.720981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79c1c-9cb5-4b09-a0b7-a7ca482d13f3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:14.747059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:14.758718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:14.759396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:14.759673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:14.761495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:14.761495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:14.761495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:14.762486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d636-18b6-46c2-856b-9e401668d7d4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:14.772327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 239/480] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:15.890797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9780f9-333e-4375-93ef-317915ac1e12 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:15.892331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9780f9-333e-4375-93ef-317915ac1e12 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:15.893105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9780f9-333e-4375-93ef-317915ac1e12 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:15.896474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 242/481] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:15.905260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:15.906020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:15.906020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:15.928070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:15.929270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:16.029522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 240/482] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:16.109538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9780f9-333e-4375-93ef-317915ac1e12 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:16.367312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9780f9-333e-4375-93ef-317915ac1e12 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:16.413334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:16.423710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:16.424589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:16.424864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:16.425210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:16.448093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ba272b65-44f3-4e78-99a2-593f6f357e48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:16.457018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:16.459748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:16.461310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:16.463567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 243/483] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:16.473362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:16.650057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be5e7-9dfb-444f-b0a0-656d21c9fa01 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:16.656650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b33e2-7365-4b10-8dd3-ae36c6e7bb33 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:16.656912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b33e2-7365-4b10-8dd3-ae36c6e7bb33 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:16.657246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b33e2-7365-4b10-8dd3-ae36c6e7bb33 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:16.660481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 244/484] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:16.665209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:16.665467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:16.665879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:16.683434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:16.691513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:16.692807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:16.728985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:16.729695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:16.729976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:16.730334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:16.731142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:16.731449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:16.732889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:16.760214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 241/485] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:16.767394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:16.846894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 245/486] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:16.896859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:16.994622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10a485-7eed-427f-8f59-7d68ec2b68c0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:17.037739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:17.039243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.039749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.040250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.056029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7170c4-680a-484f-b566-d8d6f8b747a4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:17.061902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:17.064906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:17.084209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1129ad69-9dec-4c2b-8169-a08af6232820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:17.092166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:17.093303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:17.094160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:17.094160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:17.094160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:17.094160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:17.094947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 242/487] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:17.099735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13dce5-d266-40d8-996b-6d9f23459e93 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.099735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13dce5-d266-40d8-996b-6d9f23459e93 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.099735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13dce5-d266-40d8-996b-6d9f23459e93 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.102771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:17.103663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.103946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.104219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.106525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 243/488] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:17.107931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13dce5-d266-40d8-996b-6d9f23459e93 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:17.118130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-cb05028e-c856-40d2-9a33-7d8828c99735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:17.128782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:17.130763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:17.132661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:17.138364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 246/489] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:17.138364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.140808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.140808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.159943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:17.161153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:17.168427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:17.262553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 247/490] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:17.264039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:17.320452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13dce5-d266-40d8-996b-6d9f23459e93 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:17.403839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200263-c67a-4943-afac-05814edce25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:17.405004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200263-c67a-4943-afac-05814edce25d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.405494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200263-c67a-4943-afac-05814edce25d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.405984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12200263-c67a-4943-afac-05814edce25d None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.409872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12200263-c67a-4943-afac-05814edce25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:17.409872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12200263-c67a-4943-afac-05814edce25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:17.411546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12200263-c67a-4943-afac-05814edce25d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:17.433734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 244/491] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:17.435353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200263-c67a-4943-afac-05814edce25d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:17.674556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e45c3-20bc-4d72-95fe-dc0e22b4a3ea None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:17.724307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:17.770774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:17.797427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12200263-c67a-4943-afac-05814edce25d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:17.800455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:17.803280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.804342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.805891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.810941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:17.812688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:17.817178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:17.847568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 248/492] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:17.848467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:17.896515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:17.897713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.898184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.898685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.913187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-266f1036-75cf-4299-90d0-57ebb429ba1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:17.922084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:17.923073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:17.923850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 245/493] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:17.925267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5727-653c-4c35-bdfe-d21bf32ee01e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:17.925612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5727-653c-4c35-bdfe-d21bf32ee01e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:17.925992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe5727-653c-4c35-bdfe-d21bf32ee01e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:17.927754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 246/494] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:17.928639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5727-653c-4c35-bdfe-d21bf32ee01e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:17.998120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202055f5-3936-4003-b12a-a37fa42af397 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.052345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.053303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.053640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.054088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.063521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.065466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-298d988f-968a-4fe0-a700-b44a790b5893 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:18.083670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:18.085496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:18.087284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:18.088260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe5727-653c-4c35-bdfe-d21bf32ee01e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.089068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 249/495] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:15:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:18.089068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d68b3-8f3d-4995-9015-1cff2d32b6e1 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.089068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d68b3-8f3d-4995-9015-1cff2d32b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.089068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d68b3-8f3d-4995-9015-1cff2d32b6e1 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.089068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 250/496] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:18.090504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.090504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.090504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.110427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:18.111478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:18.134455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.135393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.135755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.136107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.145580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.149716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.151279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:18.171305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:18.179239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 247/497] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.180842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.212148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 251/498] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:18.213476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.309527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f51b4-7ddd-469a-8310-da5eece66ec7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.354091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.354790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.355119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.356064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.356670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:18.356953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.358405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18c9e3-1933-4944-b590-033a2b01885d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.365551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a73d5-a022-49ac-9f7a-86bd593ed258 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.371640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 248/499] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:18.373464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.373912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.374415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.396196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:18.397554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:18.413211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.413974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.414365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.414707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.415759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:18.416267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.418717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.463914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 252/500] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:18.465342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.475427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.488047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 249/501] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:18.488974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.499308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.607780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d718e-f0b8-4f7c-8132-25e03e039b6a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.621177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f83cc0-8542-4b7f-bfa1-f05e6ec15c55 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.655022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.656076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.656513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.656953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.658170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:18.658702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.660870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.670443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.671484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.671813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.672135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.677104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 253/502] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:18.678224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.689203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.691909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:18.718367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:18.719969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:18.721427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:18.721427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:18.721427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.721427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:18.721427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 250/503] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:18.721728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.785771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.795671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:18.815865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359fb6e-a656-43b6-aa25-f1ad698c253c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.846300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14d29-4641-4967-bccf-01d12452846b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:18.865448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.866853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.867432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.867909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.886157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:18.887381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:18.887692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.887784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:18.888066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:18.890547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:18.923096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:18.924696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:18.925869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:18.925869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:18.925869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:18.925869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:18.925869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 254/504] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:18.927149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:18.961842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:18.963389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.015755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.057994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7d95-0b5c-4902-a43f-5f800a6d668c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.093573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 251/505] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:19.093573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.115193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.116442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.116897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.117340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.120332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.135768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:19.149379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:19.150580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:19.151377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:19.153022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 255/506] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:19.153403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.225993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2150b91-4c82-477a-8ebb-45f03535c59e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.266436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.267365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.267706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.268120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.277430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.278675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a90a-e269-4038-a07d-d62ef9950b2d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.279062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:19.297993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:19.299950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 252/507] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:19.301683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.319287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.319950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.320229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.320365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.320818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.321487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:19.321756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.323160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.336187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 256/508] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:19.337161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.402273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.448533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7dbf-0237-42c1-bee3-c71e15b4820a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.465643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5ebaf-a052-4cd0-927f-1588fdfc9b83 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.502109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.503065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.503474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.503829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.504687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:19.505069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.506596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.515045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.516113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.516438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.516753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.521472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 253/509] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:19.522890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.530017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.531764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:19.561206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:19.562860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 257/510] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:19.564642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.617584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.671837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803c9bb-6370-4af0-9541-ba9c7e30b74f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.703797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814623f-1483-42be-a19b-47bf3ffa21c9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.713518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.721605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.722853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.723326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.727132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.727132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:19.727132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.728360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.741083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.741745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.742022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.742324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.748205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 254/511] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:19.748425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:19.748425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.748721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.748863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.768805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 258/512] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:19.769816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.887139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc7cbc-8725-472a-b1f3-6cf0e36703ad None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.911768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4db6d9-7c31-4be0-8611-afe25d13cc91 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:19.923732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.924476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.924837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.925165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.934945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.936488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.944470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:19.953519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:19.964252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:19.964996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:19.966760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:19.969637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:19.971063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:19.971063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:19.971063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:19.971463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 259/513] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:19.971463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:19.971796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:19.988719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:20.081957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed465cab-efa0-4423-a7af-d37c0ad6e9dd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.083159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 255/514] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:19 2022] POST /identity/v3/users => generated 247 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:20.084506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.148045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.148894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.149234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.149607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.150488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:20.150933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.152619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:20.171223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 260/515] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:20.173180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.278357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5e92-71b9-42d3-8715-da288eee57e3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.339380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:20.344235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.345107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.345394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.345828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.361966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.364140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:20.367188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3846dfa-c07c-4f79-a1a5-085829604111 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:20.396279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:20.397949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:20.399688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:20.399688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:20.399688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:20.399688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:20.399688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 256/516] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:20.400819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.428665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.429409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.429824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.430332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.469612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:20.517267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.518879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:20.584635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2add3c-1102-428c-95d3-b333857724b3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.628932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.629487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.629768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.630038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.630927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:20.631232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.632692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:20.673619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 257/517] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:20.674518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.691597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:20.719855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 261/518] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:20.721946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.813484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:20.826597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64fe42-69c3-485c-a9ee-ec6d73146a96 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.866490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.867377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.867748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.868087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.876836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.878782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:20.895771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:20.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:20.899544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 258/519] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:20.900690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:20.929316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e412-c77c-4b30-ba65-d3a7441f8d2b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:20.991240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:20.992038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:20.992623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:20.992623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:20.993474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:20.993666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:20.995169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.020871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 262/520] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:21.020871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.088281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c6366-19a2-4b68-b5eb-e7204bd9c58d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.108051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.133104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.133961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.134404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.134864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.167298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.210453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40758ce-57c3-4093-a484-174caacee927 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.233628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.236469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.254824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.255638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.255970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.256228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.270503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.272081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:21.292034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:21.294348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 263/521] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:21.296819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.385131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 259/522] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:21.387087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.463537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.488235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.499171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93c9fb-8556-455a-89ef-5960a9cbcb96 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.550853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343cad1-2776-4bf0-bc26-170c5f3ff5e7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.555482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.556628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.557109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.557580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.559786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:21.560330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.563260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.601049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.602031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.602480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.602895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.609801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 264/523] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:21.609801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.618580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.620270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:21.639767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:21.641516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 260/524] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:21.643912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.778353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f9dc8-5699-452f-8776-daee1400d087 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.781801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d841a3-daad-47d5-81d5-93e1a3ead189 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:21.793863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.823639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.824842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.825121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.825391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.826130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:21.826523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.827976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.829921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:21.830761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:21.831035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:21.831308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:21.841584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 265/525] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:21.842253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:21.849493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:21.851006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:21.864838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:21.881740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:21.883532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:21.885222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:21.885222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:21.885222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:21.885222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:21.888735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 261/526] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:21.889724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.004782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1756c-8bf2-4b30-a51b-4231ce38fd02 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.016920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706b964-1bcf-4ddf-984f-338b352b1884 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.046175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.046855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.047133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.047406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.048234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:22.048548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.050002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.056288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 266/527] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:22.056937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.076505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.077566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.078037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.078668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.079576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:22.079890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.081467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.103053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 262/528] 10.222.0.48 () {56 vars in 1139 bytes} [Fri May 27 09:15:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:22.109219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.141034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.214306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.242735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291acfb-bacc-437e-abf5-54d762c3e037 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.246961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f3d3c-c106-438d-a3f8-83d55fbba7f5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.290639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.291444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.291727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.291996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.293350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:22.293753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.295278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.313546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 263/529] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:22.314902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.323065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.323912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.324189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.324463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.332396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.334072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:22.351424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:22.352350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:22.353168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:22.358364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 267/530] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:22.359399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.438576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.452875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489720bc-852a-4f90-9d34-b058d97be7fc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.501331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.502025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.502349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.502623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.531574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fa484-7895-4c33-944d-443b120a0ca3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.581585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.582416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.582694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.582972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.588896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.591095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bd977-6c0e-47db-8d67-574a19237e3c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.591326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.592774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.595836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:22.608786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:22.610048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:22.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:22.612330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 268/531] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:22.613214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.724307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.742798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 264/532] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:22.743952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27bf12-16c8-40ee-8220-b3cd018b408f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.744237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27bf12-16c8-40ee-8220-b3cd018b408f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.744513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27bf12-16c8-40ee-8220-b3cd018b408f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.745799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 265/533] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:22.746517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27bf12-16c8-40ee-8220-b3cd018b408f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.751632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23769-93f3-413d-908a-c7c49cb7f6c8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.799937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.800599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.800876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.801146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.801933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:22.802266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.803711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.823911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 269/534] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:22.828933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:22.879280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27bf12-16c8-40ee-8220-b3cd018b408f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:22.920860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:22.921524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.921806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.922068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.922885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:22.923192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:22.924653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176c477-fb7f-46f2-9496-0734a7b166a3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:22.934744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:22.951332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 266/535] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:22.951332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:22.951332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:22.951332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:22.966234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:22.967454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:23.004221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214407d-063d-48ba-b03e-d2c0547c3905 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.012565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:23.047261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 267/536] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:23.047261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.059694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.061196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.061540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.062003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.078847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.080786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:23.103050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:23.104006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:23.105657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:23.105657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:23.105657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:23.105657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:23.105657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 270/537] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:23.106398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.167760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171c7e-53ed-4a0a-ba3b-53de8660b2f6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.218524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.219475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.219475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.219738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.220573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=3963b8bacbc74ef2a579be684a219ac2) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:23.220882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.222400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.229278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 268/538] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/role_assignments?user.id=3963b8bacbc74ef2a579be684a219ac2&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:23.230747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.231709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa2ff-ec56-421a-8fdf-dcf7f211cd71 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.258856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:23.289115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.289785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.290061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.290367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.291488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=ab8e09e92e284747b80ca816b6e15a2c) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:23.291796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.295059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.305154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 271/539] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/role_assignments?user.id=ab8e09e92e284747b80ca816b6e15a2c&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:23.306571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:23.307183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.449371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d012f-5c80-488b-9d44-a7547ed90c79 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.496564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e385a-2e92-4e4a-a976-28023c50ae29 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.537072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.537745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.538029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.538511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.539244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:23.539357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.540777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.571986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.572727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.573149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.573428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.577635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 272/540] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:23.583443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.599075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:23.608833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:23.609777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:23.610599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 269/541] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:23.619304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.646388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:23.779113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876795cd-f91b-46e2-b119-30b6d56de76b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.831969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.833473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:23.835382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.836282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.837045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.837428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e950-814f-48cf-b311-be83956cfa8d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:23.854759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.864602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.879222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:23.879941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:23.880200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:23.880708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:23.881665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:23.882011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:23.883606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:23.898041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:23.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:23.900133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:23.900709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 270/543] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:23.900825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 273/543] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:23.900927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:23.900996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:24.034529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fade7-4a69-4ad2-864c-d58a43755bbd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:24.075178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:24.075985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.076265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.076544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.094228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:24.094228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:24.110620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:24.111525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:24.112340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:24.112340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:24.112340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:24.112340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:24.112340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 271/544] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:24.113617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:24.118439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:24.125090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c476f4-57c6-4dec-b3ca-25d40320646c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:24.149773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:24.208106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:24.209334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.209836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.210357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.211919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:24.212397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:24.215807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:24.267623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 274/545] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:24.275143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:24.411412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002ae6d-9600-48ff-95b7-2635cb017e7b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:24.467756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:24.469254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.469975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.470497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.472201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:24.472731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:24.474907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:24.490578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 272/546] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:24.496453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:24.534609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9518b8-7db2-46b4-ae33-17db8cf727ac None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:24.590637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:24.613925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:24.615780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.616348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.617126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.618447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=07600562586a47a696ec87ef60838559) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:24.619315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:24.622853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadaad7-e16e-427b-9278-000ce4c09bb9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:24.634820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 275/547] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:24 2022] GET /identity/v3/role_assignments?user.id=07600562586a47a696ec87ef60838559&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:24.640812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eba3b-8e21-433b-a664-ecedf9f5dca9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.641131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eba3b-8e21-433b-a664-ecedf9f5dca9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.641465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eba3b-8e21-433b-a664-ecedf9f5dca9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.646552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 276/548] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:24.656970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.657387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.657825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.665443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dcb3e-19b2-41dd-8fed-6d457a85a3a4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:24.711940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:24.713938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:24.736043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:24.805928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 277/549] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:24.810161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:24.811380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:24.811863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:24.812369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:24.822225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:24.824147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:24.840840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:24.841924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:24.842999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:24.845896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 273/550] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:24.850078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:24.855612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:25.016675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a7d2-3651-479c-9296-00756db5d529 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:25.083304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5db2b2-5583-476a-972b-e856deed17c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:25.090915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:25.091728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:25.092004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:25.092275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:25.115355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:25.123242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:25.134617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 278/551] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:25.136165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:25.158578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:25.164313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:25.166886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:25.167712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:25.168061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:25.168409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:25.169331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:25.169648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:25.171149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:25.205685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 274/552] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:15:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:25.212457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:25.331606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bd77f-05de-446f-ad68-39c621df413b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:25.384214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:25.388801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:25.389610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:25.390244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:25.391372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:25.391997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:25.397129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:25.424433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 279/553] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:25.425479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:25.440963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce8a0-0cfc-4f62-9f1a-4dcb95ce0724 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:25.483583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:25.485010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:25.485463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:25.485965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:25.557293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:25.581036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:25.581036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:25.701816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 275/554] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:25.711150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:25.731664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:25.832451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e238c-9eb4-45e4-9013-2e170a9daa53 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:25.920703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:25.924643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:25.928151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:25.929226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:25.972009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:25.974513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:25.989822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:26.009659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e7160-b6e4-4a5e-ab22-e9dafa7775c2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:26.041372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:26.043230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:26.044921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:26.044921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:26.044921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:26.044921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:26.044921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 280/555] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 619 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:26.055815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.095001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.096817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.097089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.097528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.113135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:26.115238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-2a5bc9e0-5794-401a-93ca-e0bcbe7a51d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:26.137641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:26.138835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:26.140419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:26.140419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:26.140419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:26.140419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:26.142785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 276/556] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:26.144481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facebab3-9ea7-47c7-ba24-a69277db24ed None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.145646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facebab3-9ea7-47c7-ba24-a69277db24ed None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.146120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facebab3-9ea7-47c7-ba24-a69277db24ed None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.153331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 277/557] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:26.154679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facebab3-9ea7-47c7-ba24-a69277db24ed None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.246401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b6bd-3b95-4cbf-bdff-94b762581fc1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.275993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:26.289870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.291180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.296128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.296128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.296128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:26.296128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:26.296625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252aee1-bf59-40aa-970f-26171865ae41 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:26.319815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 281/558] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:26.321759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.322844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.323282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.346412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:26.348357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:26.367730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facebab3-9ea7-47c7-ba24-a69277db24ed None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.419305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.420929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.421434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.421919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.423304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:26.423864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:26.426582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e885-4504-45db-9c50-f27d2b839a3c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:26.455999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 282/559] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:26.462248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 278/560] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:26.462248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ee09-4e4b-4700-96d5-5bc56a955ee8 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.462248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ee09-4e4b-4700-96d5-5bc56a955ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.462248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6ee09-4e4b-4700-96d5-5bc56a955ee8 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.467250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.470727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 279/561] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:26.472256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ee09-4e4b-4700-96d5-5bc56a955ee8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.477757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:26.641252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6ee09-4e4b-4700-96d5-5bc56a955ee8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.654948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03912670-76db-49e9-b9c1-7f987fee8d42 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.679513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.680313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.680595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.680875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.683440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:26.689477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:26.691018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-24aef283-43e3-4ea2-a7da-ad9794fbf1eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:26.706329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:26.708158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:26.710224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:26.712278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 280/562] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:26.713718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.714009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.714842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.723680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.724489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.724771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.725049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.734596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:26.734849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:26.735945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:26.736228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:26.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:26.754604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 283/563] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:26.756475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.834624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:26.850082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 281/564] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:26.851242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.895803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaddd30-4a21-4c15-89d1-be157f3ea105 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:26.947249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:26.948356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:26.948922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:26.949356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:26.962857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:26.971458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:26.972480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:26.973319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 284/565] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:26.974007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:26.988714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47feb63e-0754-4561-b855-fbc88458df73 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.033627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.034444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.034773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.035102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.036104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:27.036613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.036737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.038858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.054727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 282/566] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:27.056092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.146478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.150900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ceab3-5ff7-4326-99ce-d6903f4c44bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.203789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.204738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.205172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.205627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.206882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:27.207367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.209152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.219838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b038b2-f29e-46e6-a3eb-1204b21b3284 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.232143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 285/567] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:15:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:27.233347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.286287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.287500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.287919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.288339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.301857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:27.310655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:27.312523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:27.314175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 283/568] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:27.318523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.405490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.419929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70ade-e55c-4833-8891-2a0a4e5f8c9c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.465080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.465868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.466147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.466525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.467274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:27.467576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.468959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.477672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 286/569] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:27.478545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.478653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54386f0d-2f12-4ff4-96cf-b16b0f75f395 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.493997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.527951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.528768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.529155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.529514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.530875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=444436f77ef74d9fb825aafa19ccc146) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:27.531285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.533111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.553086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 284/570] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/role_assignments?user.id=444436f77ef74d9fb825aafa19ccc146&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:27.553086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.646857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f5a9-6e95-48e5-85e6-6e9ac4d9b8c6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.706233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.706956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.707245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.707527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.722609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.724208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.738016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb1964-62cf-4aa1-aa02-46f2e61d8d9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-edfa179f-848d-4454-b4f0-559cca917769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:27.752968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:27.753944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:27.755002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:27.755002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:27.755002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:27.755002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:27.759287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 287/571] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:27.760307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aabdd7-bf34-4ece-87fe-b1978c416653 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.760585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aabdd7-bf34-4ece-87fe-b1978c416653 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.760863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aabdd7-bf34-4ece-87fe-b1978c416653 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.762679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 288/572] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:27.763050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aabdd7-bf34-4ece-87fe-b1978c416653 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.773344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.794043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.794855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.795230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.795594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:27.801390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:27.801390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:27.801390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:27.810060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 285/573] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:27.811791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:27.864290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:27.927397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aabdd7-bf34-4ece-87fe-b1978c416653 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.970024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4ae0-075b-479f-aceb-5ed2e14bdeb4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:27.985649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:27.987113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:27.987731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:27.988381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.010699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.013972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.036409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.037551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.038047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.038564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.039974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:28.040522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.043746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4f18daec-0afc-4fc6-a2db-d6345af78dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:28.051231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:28.052403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:28.053424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:28.053424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:28.053424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:28.053424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:28.054261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 289/574] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:28.056333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.057000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.057497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.064044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 286/575] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:28.065272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.087898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:28.090326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:28.182371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.196275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.210232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 290/576] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:28.210232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.210540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800d25-860d-47a0-aaa6-d39fa8fb07e3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.259772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.260840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.261349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.261765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.279392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.283525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:28.319489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:28.320875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:28.321763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:28.321763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:28.321763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:28.321763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:28.326988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 287/577] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:28.326988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.334919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a40d7-08c3-4a86-83c9-665eead9d50f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.378709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.379684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.380140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.380612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.381520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:28.382000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.383601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.410509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 291/578] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:28.413271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.460780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c398a-5f8b-4375-922d-6f53ffc9b06c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.478290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.505466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.506348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.506633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.506904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.515572 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.516958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:28.532152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:28.533916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:28.536375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:28.538225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 288/579] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:28.539201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.555385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.591034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca694-36bd-4eea-9536-d851bcb46946 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.649642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.650938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.651398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.651924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.670773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:28.684227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:28.686073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:28.687758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:28.691802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 292/580] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:28.692961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.703811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4f8c-9835-429a-971e-2d05d08d4793 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.765178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.766387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.766387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.766697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.767595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:28.767950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.770140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.799448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 289/581] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:28.799448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.850841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b075c-134c-4c38-b15c-8a3ee4ddbe9d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:28.867379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.904980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:28.905786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:28.906248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:28.906737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:28.916621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:28.918656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:28.931693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:28.935633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:28.945193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 293/582] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:28.947326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:28.998795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21a81-291d-403d-8d54-21bc16e188e5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.062938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.063979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.064486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.064978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.066557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:29.067129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.069899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.071212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1617986-5f3a-47dc-bb0e-8fc245ee63d7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.102251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 290/583] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:29.102251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.134889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.136122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.136784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.137316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.139240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:29.139240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.140987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.156292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 294/584] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:29.157742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.216770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:29.256373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb475e-931f-40b4-ba60-978f5badcc17 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.310617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:29.321563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.322474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.322808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.323133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.335842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.338501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.346893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e5e5a-c71e-477b-9cd2-1f60f5eb4cb1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:29.369466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:29.371852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:29.373946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:29.376910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 291/585] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:29.378569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.405251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.407909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.407909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.407909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.407909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:29.408948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.410931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.446011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 295/586] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:29.447879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.549537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c4335-7c96-4e7d-b631-1295ed88359d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.598233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.599507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.599924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.600348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.618177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=55612ce26632413ebd0dfd5fc081b656, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.620083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.629988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:29.650891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd8043-c83e-4d90-a168-da9f8b6ef2fe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.694898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:29.696044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:29.696323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:29.696603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:29.703010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 292/587] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:29 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:29.705216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.722727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:29.724607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:29.747524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:29.748624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:29.749670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:29.749670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:29.749670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:29.749670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:29.752674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 296/588] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:29.753859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:29.800529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:29.935801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea5c2c-20ca-4d17-b82d-e94b9afc4e20 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:29.960741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce113b-22b3-4b42-bf38-adfd4065f100 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.012197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.013561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.014181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.014831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.016744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:30.017423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.020582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.035086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.036304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.036752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.037195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.055315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.060361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.062988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.063844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 293/589] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:30.065304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:30.099261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:30.101165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:30.103554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:30.103554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:30.103554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:30.103554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:30.106676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 297/590] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:30.116000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.260206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.266922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f23e7-420a-4eac-b773-9594bd4f925b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.308976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc41aa9-5b5d-4761-ac08-b27302476ac5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.332784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.333896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.334415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.334845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.336147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:30.336688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.339322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.353846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.354712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.355546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.355830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.367863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 294/591] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:30.367863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.368298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.369378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3d9b4037-bd00-479b-a1d5-8d94bc452be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:30.389712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:30.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:30.394325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:30.394325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:30.394325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:30.394325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:30.398808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 298/592] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:30.400370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793999fe-4a50-40d5-b189-3cd6a6beb109 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.400898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793999fe-4a50-40d5-b189-3cd6a6beb109 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.401459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793999fe-4a50-40d5-b189-3cd6a6beb109 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.404211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 299/593] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:30.408429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793999fe-4a50-40d5-b189-3cd6a6beb109 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.471757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.567295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816e89e-86d4-4afc-8066-3ecdf6b0e0a9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.567478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793999fe-4a50-40d5-b189-3cd6a6beb109 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.607679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.608435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.608802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.609306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.609948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:30.610403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.611934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.621084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.622037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.622490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.622895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.633985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 295/594] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:30.633985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.635468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.637710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ad45ddef-b9a6-4b2f-88a2-26c68106ec41 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:30.655265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:30.656495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:30.657683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:30.658374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.658460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 300/595] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:30.659606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.659886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.660156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.683366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:30.685130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:30.771347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 301/596] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:30.772460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.792838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddc2df-0d24-4ba6-81ab-236de4153b21 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.825254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.835703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.836382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.836649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.836979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.837726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:30.838034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.839541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.855354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 296/597] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:30.856225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:30.921993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe73ecc-cda1-4185-b412-8e4bb14094f7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:30.961950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:30.977266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:30.978059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:30.978372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:30.978647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:30.987288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:30.988746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:30.999087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45615ba-05c8-4d4c-a922-b1f2cdde3c8c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:31.002833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:31.004424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:31.006155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:31.009024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 302/598] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:31.009957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.046318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.047098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.047346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.047553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.048390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:31.048707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.050168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.071989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 297/599] 10.222.0.48 () {56 vars in 1139 bytes} [Fri May 27 09:15:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:31.073004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.148414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.157425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144e98-0629-4e37-9e0e-bb500025aee5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.203813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.204744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.205086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.205417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.218722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:31.220175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952396-9ae2-4a74-9a5a-6e8cd4f5fdf0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:31.227133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:31.238529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 303/600] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:31.240416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.282605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.283402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.283682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.283952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.287364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.292212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.294304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:31.316097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:31.317104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:31.318276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:31.318718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 298/601] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:31.320017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.368448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16216-18ec-430b-b0eb-0e3ff6cdde62 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.421942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.421942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.421942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.421942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.422743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:31.422743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.422743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.437930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 304/602] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:31.437930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.459172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.476484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9237e-6309-41ca-adf5-49d6818d40b2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.535207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.536565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.537077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.538797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.553500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.555169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.569970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2e7dc-0a1b-434d-bccf-d07e93540a31 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:31.578486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:31.579731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:31.581300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:31.581300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:31.581300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:31.581300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:31.582291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 299/603] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:31.582994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.617044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.618013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.618337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.619022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.620352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:31.620931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.622369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.635693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 305/604] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:31.635693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.647473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.713193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4d9b1-c335-4517-9ccb-d17013461561 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.735683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.762104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.763527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.763989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.765384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.766757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:31.767270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.769060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.769350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902a61b-7059-48de-a48e-658d5c738f23 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.796495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 300/605] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:31.796495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.829411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.830480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.830821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.831164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/ab8e09e92e284747b80ca816b6e15a2c/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.849133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=ab8e09e92e284747b80ca816b6e15a2c, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.850797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.927359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e3025-5d20-4b01-ad96-c2c03e622554 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:31.939405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:31.968709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:31.969287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:31.969564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:31.969842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:31.970600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:31.970906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:31.972313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:31.987169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 301/606] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:31.988619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:31.995396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 306/607] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:31 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/ab8e09e92e284747b80ca816b6e15a2c/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:31.997517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:32.138844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f64a38-7118-48e1-9174-8e30f7a0915a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:32.159663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f65a8-0314-44c4-82b5-bbcf0427b36e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:32.190165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:32.191325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.191709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.192212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/3963b8bacbc74ef2a579be684a219ac2/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.201181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:32.213381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:32.214489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.214909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.215192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.217424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=3963b8bacbc74ef2a579be684a219ac2, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:32.219589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:32.231284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:32.233837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:32.260900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:32.263231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 302/608] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:32.265568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:32.299198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:32.327384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 307/609] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:31 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/3963b8bacbc74ef2a579be684a219ac2/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:32.329285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:32.488985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38ad34-cf11-4929-bc07-cb49c37b9fae None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:32.557254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:32.559028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.559339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.559592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.578566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa0123-fd2f-4cfa-8e50-e44ba323d2db None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:32.580681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:32.584716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fb59b231-1d91-4c5b-bdca-0d463b2778c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:32.611549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:32.613135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 303/610] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d944da-607a-40f6-86db-cab47e4d34d5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.619489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d944da-607a-40f6-86db-cab47e4d34d5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.621869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d944da-607a-40f6-86db-cab47e4d34d5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.621869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 304/611] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:32.623008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d944da-607a-40f6-86db-cab47e4d34d5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:32.634994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:32.636238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.636603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.636947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.637776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:32.638158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:32.639686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a168526-ab06-4c55-8dfe-118251e477c1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:32.654309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 308/612] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:32.655900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.656389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.656984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.692356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:32.695154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:32.755243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:32.794423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:32.810669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 309/613] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:32.811892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:32.846678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d944da-607a-40f6-86db-cab47e4d34d5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:32.892392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:32.893235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:32.893615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:32.893987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:32.894937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:32.895369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:32.897115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:32.918232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 305/614] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:32.918232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.016967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219965b-4c8c-4946-94f7-ab1f6644f428 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.058709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.059809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.060243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.060665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.068864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:33.071024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:33.087863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:33.089000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:33.090738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:33.093091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 310/615] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:33.094097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.114477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c420a-1bd9-4979-88bd-905b8f1976fc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.150934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:33.177727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.179374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.179747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.180087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.199713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:33.208653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:33.223441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:33.225435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 306/616] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:33.228671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.242730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4ff4-1bc0-4024-a440-cddde8c226f9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.314233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.317404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.318498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.320834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/07600562586a47a696ec87ef60838559/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.356709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=07600562586a47a696ec87ef60838559, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:33.358181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:33.371097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f96b-7682-4e37-a800-f2b10424c4cd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.432616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.434320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.434792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.435294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.436938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:33.437443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:33.440204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:33.474911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 307/617] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:15:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:33.475781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.479860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 311/618] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:33 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/07600562586a47a696ec87ef60838559/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:33.488139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.650848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:33.707030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:33.749704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24ca67-8eff-4687-868f-1c5af4e14891 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.829341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.830371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.830658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.830928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.831401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a4ca4-1eef-4ca8-a3f9-86572dc141c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:33.844589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:33.846109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:33.867823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:33.869588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:33.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:33.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:33.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:33.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:33.871518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 312/619] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:33.877224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:33.906674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:33.907977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:33.908325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:33.908605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:33.910964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:33.912771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:33.915932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:33.950558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 308/620] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:33.960089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.134255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f21bac-9ec5-40eb-abd7-297c1f6fb938 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.198818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:34.204474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaace1b-1b9d-48a6-976d-2051fdd64be1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.217075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.217748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.218025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.218318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.219044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:34.219361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.220726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.243300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:34.249373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 313/621] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:34.254656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.265259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.267133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.267433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.267711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.298754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.300236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:34.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:34.340359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:34.341248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:34.341248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:34.341248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:34.341248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:34.341248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 309/622] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:34.347220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.394003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0e520-8fbe-4741-afc4-6f5071370bfe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.433933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.435217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.435745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.436264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.445963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.447847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:34.466706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:34.467697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:34.468561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:34.471658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 314/623] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:34.479203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.541282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:34.602758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b8fd1-1285-4b27-985d-07f5314f5203 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.604115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c24791-7924-43c8-984c-cc0b11b3926a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.643579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.644704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.644926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.645194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.645987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:34.646325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.647767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.664640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 315/624] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:34.670819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.683089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.684658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.685165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.685635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.686601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:34.687111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.688780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.718860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 310/625] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:34.721678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:34.785120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:34.787424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e219-333c-4c27-909e-d9119f2deba8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.798222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:34.826629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.827545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.828016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.828455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/444436f77ef74d9fb825aafa19ccc146/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.844974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=444436f77ef74d9fb825aafa19ccc146, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.846477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94088c14-6c0d-45b0-bab0-12dbfe05c0c4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.865560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc3589-74cc-4cd0-851d-c989578360f3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:34.907949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:34.908738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.909020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.909294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.919376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:34.921631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:34.938415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 316/626] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:34 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/444436f77ef74d9fb825aafa19ccc146/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:15:34.940035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168d373-281d-4125-8e15-88aa4aa58141 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.946569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168d373-281d-4125-8e15-88aa4aa58141 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.947042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168d373-281d-4125-8e15-88aa4aa58141 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.954238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 317/627] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:34.966059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:34.968090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:34.969506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:34.973636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:34.973907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:34.974191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:34.977490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 311/628] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:15:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:34.983521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.035477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:35.036716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:35.103618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:35.114747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b115-0040-48d1-8ffa-31a6c3022e3d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:35.159132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:35.161258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.161798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.163261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:35.164517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:35.166583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:35.168383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:35.215097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 318/630] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:35.215373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 312/630] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:15:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:35.217735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.258049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.341881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:35.358709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b120e-dfac-4bef-b328-bfed1a17432a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:35.408465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:35.409127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.409477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.409675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:35.426144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10407f-afc3-4b0f-b1ad-8733e52f6fca None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:35.526303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:35.527202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.527486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.527758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:35.533410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:35.534966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:35.567300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:35.584633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:35.585989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:35.587126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 313/631] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:35.592855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.632034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:35.641739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:35.657911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 319/632] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:35.670884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.787366 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60dfc2-849b-48b6-8371-f7911beb7a75 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:35.831741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:35.833651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.835523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.835990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:35.837177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:35.838823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:35.841258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb13f4-d583-4a61-910c-1fdff65c4569 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:35.857264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 314/633] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:35.860191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f1f4c-005a-476d-8a69-8a4d7da10e42 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.860755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f1f4c-005a-476d-8a69-8a4d7da10e42 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.861469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f1f4c-005a-476d-8a69-8a4d7da10e42 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:35.863730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 315/634] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:35.865949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f1f4c-005a-476d-8a69-8a4d7da10e42 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:35.924968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013242e-a445-4cd9-9a26-d0f2f1fc582b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:35.969804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:35.993453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:35.995188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:35.996094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:35.996798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.015393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f1f4c-005a-476d-8a69-8a4d7da10e42 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.017902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.022326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.054746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.055532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.055810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.056080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-87bb15e6-56ed-4f3d-867a-54af80e188c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:36.060193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:36.062260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:36.064171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:36.064171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:36.064171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:36.064171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:36.067914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.068511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 320/635] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:36.069385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.071806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.072775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.073541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:36.090077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:36.091849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 316/636] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:36.093584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.109796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:36.112533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:36.200779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:36.229359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384309-6f15-4d35-8a2b-e1b8b206b471 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.255039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 321/637] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:36.255039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.274410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:36.302393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.304052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.304866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.305678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.307422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:36.308394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.311494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.337080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 317/638] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:36.339114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.392502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdb7b-359f-46fb-8edf-8bd7e118eed2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.442448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.443231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.443514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.443786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.444533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:36.444841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.447341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.463855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 322/639] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:36.465334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.512836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:36.557846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd62ec-515f-40f1-8ab7-4a1ee8b36a52 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.610324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b9a86-c3ee-438e-bea8-9f0efe7ba51e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.636126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.638706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.639542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.640331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.655857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.656703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.656985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.657262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.666015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:36.675336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:36.676136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:36.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:36.680049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:36.681782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 318/640] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:36.682737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.685943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:36.715557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:36.717357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:36.719185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:36.723718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 323/641] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:36.725711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.741674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:36.813454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb597b8-8f81-4652-b5b0-d007d4679b57 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.847677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:36.852695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.853731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.854050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.854395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.860985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98564ac-b85c-41a2-92b9-b8e3d5fe960c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:36.865202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.866709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:36.882706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:36.884819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 09:15:36.887633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:36.889992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 319/642] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:36.891249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:36.908013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:36.908735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.909021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.909321 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.910053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:36.910423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:36.911798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0185d-bd60-4283-a8a7-85b7e019c65e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:36.922366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 324/643] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:36.923862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056563f0-e53d-4183-9e81-4371e6ea8406 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:36.924173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056563f0-e53d-4183-9e81-4371e6ea8406 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:36.924417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056563f0-e53d-4183-9e81-4371e6ea8406 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:36.926138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 325/644] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:36.926792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056563f0-e53d-4183-9e81-4371e6ea8406 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.030610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fc89e-0476-42c6-b42f-d54d3b4193b9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.034162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:37.064720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056563f0-e53d-4183-9e81-4371e6ea8406 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.088301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.089153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.089830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.090152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.091604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:37.092045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:37.094624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:37.115019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 320/645] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:37.116208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.131155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.132260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.132715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.133159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.135119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:37.135636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:37.138633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb088b5-5ad2-4926-9a60-ecf13477d144 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:37.176425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 326/646] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:37.176425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.176425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.176425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.200607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:37.202676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:37.207617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:37.295005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9afe3e-e360-44b4-847e-3eb1258e5414 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.317893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 327/647] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:37.319178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.362939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.363746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.364042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.364299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.378581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:37.380086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:37.399489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:37.401238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:37.403226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:37.403226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:37.403226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:37.403226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:37.405328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 321/648] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:37.406302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.428260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:37.523380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b05b7-0c1d-44bd-9f82-0dc841b647a4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.534705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746c70-8bf9-4800-a01d-168860b595ff None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.583294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.584428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.584930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.585481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.598386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.599531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.599990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.600444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.602039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:37.603192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:37.605533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:37.623693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 328/649] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:37.623693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.626418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:37.756793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:37.760021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbf42e-49a2-4861-94f2-df15ca2f760e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:37.796668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:37.843289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c554598-ca23-4601-8bc5-196705752373 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:37.895902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:37.896837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.897743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.897743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.911549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:37.920854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:37.922048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:37.923104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:37.926097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 329/650] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:37.926097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:37.952404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 322/651] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:37.954245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f900c0-7d30-4900-aad7-29d87555f03d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:37.954703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f900c0-7d30-4900-aad7-29d87555f03d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:37.955403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f900c0-7d30-4900-aad7-29d87555f03d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:37.957236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 323/652] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:37.958235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f900c0-7d30-4900-aad7-29d87555f03d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.040034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.088461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f900c0-7d30-4900-aad7-29d87555f03d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.102916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905eade-f546-480d-b732-1b75ae9c1ff4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.108649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.132393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.133230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.133703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.134167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.135071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:38.135593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.137812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684dc15-46bd-4aa6-bb4c-1d346774bfd7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.154246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 324/653] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:38.154533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.154533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.154533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.154533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.155139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:38.155286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.155444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.155612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.155612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.155877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.169756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 330/654] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:38.175209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.182079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:38.183585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:38.280328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 325/655] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:38.281704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.319210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758184c-a050-4d32-99fb-85f4bc8c55d8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.348305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.360922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.361629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.361908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.362179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.374177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.375679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:38.393717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:38.394765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:38.395654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:38.395654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:38.395654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:38.395654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:38.395654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 331/656] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:38.395962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.411550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bca0a-3433-4054-b8fe-5379938d7b1a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.423515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.463195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.464015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.464391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.464638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.475954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.477440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:38.497537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:38.498673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:38.499676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:38.499676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:38.499676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:38.499676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:38.503324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 326/657] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:38.504269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.524262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c98c5-e24c-4b8c-a922-48552a87d350 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.570382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.571476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.571871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.572147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.580806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.582410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:38.598983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:38.600229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 332/658] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:38.601581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.643310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.659595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3b93f-35b3-4a44-acfd-c4142c6ce49d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.702129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.702959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.703316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.703519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.717383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:38.725164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:38.727022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:38.728604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:38.729878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f892-aa14-4dc5-86b1-6d2f8a5121ae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.731927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 327/659] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:38.732857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.757523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.794962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:38.795645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:38.795926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:38.796199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:38.796939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:38.797246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:38.799020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:38.812438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 333/660] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:38.813309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:38.872384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74abb8-5464-4b3d-8e03-44c8ee9c749f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:38.940142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:38.949680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7df60-1136-4918-89ed-8c04b5d978b6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.000505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.001298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.001647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.001991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.002958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:39.003342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.004906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.034300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 334/661] 10.222.0.48 () {56 vars in 1137 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:39.035661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.043606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.044272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.044626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.045433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.046179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:39.046512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.048389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.064053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 328/662] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:39.065164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.165491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.188242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0933091-eb2c-48b4-a2c5-46d0e28c0651 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.188565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e86c-cdd5-43f9-a608-c2883964d9a0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.227230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.228072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.228319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.228683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.236510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.237967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.240707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.241665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.242163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.242664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.244093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:39.244642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.247114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:39.252420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:39.253364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:39.254808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:39.258698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 329/663] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:39.259296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.266240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 335/664] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:39.266240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.287772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.388422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c1b12-a2a4-46fb-a05a-662034670f43 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.405599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc6b7b-c22b-4705-ab09-374b2e239df0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.434341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.435476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.435987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.436486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.447101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.448232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.448232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.448427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.456106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.457867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.460086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:39.475433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:39.476638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 09:15:39.478143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:39.482384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 336/665] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:39.483214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.555048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.556604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.588825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.631393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b5695-f988-4a41-b879-3903f3fcee96 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.670470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 330/666] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:39.671144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.684779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.685459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.685747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.686046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.686869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:39.687183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.688660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.708313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 337/667] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:39.708313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.725834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.811924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbb783-dc44-4957-80a9-ea02fbf0fda5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.837925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a18ba-de6c-4677-b58c-7d03e51c8535 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:39.858472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.859866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.860252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.860663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.877985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.879714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.895056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:39.895752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:39.896034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:39.896307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:39.897189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:39.897505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:39.899058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:39.900688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:39.902672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:39.904712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:39.904712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:39.904712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:39.904712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:39.907291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:39.907925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 331/668] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:39.909054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:39.928393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 338/669] 10.222.0.48 () {56 vars in 1139 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:39.929768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.041088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.053126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ece3a-4650-4ed1-9344-7c25d2a6bd69 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.062782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244ecc-93eb-4657-b40e-69d105690fbb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.099564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.100459 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.100932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.101211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.114164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.115229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.115810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.116097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.116481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.116770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.117610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=ab8e09e92e284747b80ca816b6e15a2c) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:40.118094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.119377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.128009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 339/670] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/role_assignments?user.id=ab8e09e92e284747b80ca816b6e15a2c&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:40.129192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:40.137929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:40.139147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:40.140334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:40.140334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:40.140334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:40.140334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:40.142530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 332/671] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:40.144027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.232210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.277107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6929f-8a26-4b8e-a42e-891ace1afe7a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.298105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce17c-471e-479f-a59c-0b67801e32c1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.326416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.327387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.327831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.328265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.354325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.355883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.356288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.356644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.357788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:40.358731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.360298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.366622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.376537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 340/672] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:40.377411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.441122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.442724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.529164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7692469-1aae-4164-95e0-bf0c28a7711e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.556252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.581968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.582723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.583079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.583428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.584402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:40.584788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.586697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.603568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 341/673] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:40.604306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.612795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 333/674] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:40.618374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.673810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.739940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba0610-875b-49ce-8cb9-969a0e77f789 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.746426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ebba-9c4a-4a87-9184-805cb4af27bd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.780593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.782001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.782930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.783689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.795135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.795346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:40.796124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:40.796467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:40.796805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:40.797536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.804813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:40.806432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:40.816993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:40.818221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:40.819391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:40.822011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 342/675] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:40.823254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:40.824517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:40.825408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:40.826336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 334/676] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:40.832353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:40.882800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:40.960324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393516d2-bf29-4dae-90c1-4ba92c9af7ae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.972698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158379e-1bae-49c3-837e-3bef15ed780f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:40.975688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:41.000055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.001257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.001762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.002261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.013144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.013883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.014166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.014472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.015401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:41.015720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.016601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.017252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.018598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.038253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 335/677] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:41.038782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:41.039662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:41.040563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:41.040563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:41.040563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:41.040563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:41.040801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.042702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 343/678] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:41.048681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.219335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:41.223679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bee058-36cb-4df7-8abd-7cc8cfe704fc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:41.236799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918b1-4e33-420a-bc81-c8094758f920 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:41.283445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.284448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.284894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.285341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.286862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:41.287502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.289605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.295606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.296593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.297034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.297472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.318750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 336/679] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:41.318750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.357388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:41.431246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.434983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.455411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8066ca-a974-4ee8-8c64-661aa08858ec None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:41.499377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.500268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.500609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.500951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.501744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:41.502106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.504183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598461-bc01-49f3-ac0f-d574f5524503 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.518330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 337/680] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:41.520596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d5db-f1ad-4807-abf5-7fa3b07e18df None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.521079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d5db-f1ad-4807-abf5-7fa3b07e18df None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.521555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d5db-f1ad-4807-abf5-7fa3b07e18df None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.524284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 338/681] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:41.529571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622d5db-f1ad-4807-abf5-7fa3b07e18df None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.606853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:41.620073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 344/682] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:41.621521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.695223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:41.725127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622d5db-f1ad-4807-abf5-7fa3b07e18df None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:41.775986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.776599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.777419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.777694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.778782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=3963b8bacbc74ef2a579be684a219ac2) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:41.779094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.780859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d29414-ca48-4b22-b286-fb3cb0ac2537 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.788749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 339/683] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/role_assignments?user.id=3963b8bacbc74ef2a579be684a219ac2&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:41.791026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744590-2323-4e29-b394-e66f78b8937c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.791507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744590-2323-4e29-b394-e66f78b8937c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.791995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744590-2323-4e29-b394-e66f78b8937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.806286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e8672-cffc-4eab-994e-f7098d0e4844 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:41.822139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-25744590-2323-4e29-b394-e66f78b8937c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:41.825092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-25744590-2323-4e29-b394-e66f78b8937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:41.879678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:41.881711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:41.882817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:41.883373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:41.898248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:41.901641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:41.932393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:41.934781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:41.937107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 345/684] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:41.938726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:41.974493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 340/685] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:41.977398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25744590-2323-4e29-b394-e66f78b8937c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.075402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:42.162188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:42.177682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00f253-ab32-4f51-821a-73829dc5309e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.217123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25744590-2323-4e29-b394-e66f78b8937c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.233072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.234629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.234975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.235321 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.252318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:42.254434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:42.281394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:42.283180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:42.284933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:42.284933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:42.284933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:42.284933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:42.290365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 346/686] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:42.292071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.296609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.298663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.299457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.301706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.303640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:42.304512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:42.307569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:42.335569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 341/687] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:42.337169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.516795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b30ab-68d8-49e7-b086-3e32ba5c340c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.521065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:42.590768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a342b00-4698-4ecb-8983-f24d44833952 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.612287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.614020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.615258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.615941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.641691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:42.661728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:42.663544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:42.664964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:42.667534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.668788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.669532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.670012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.670247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 347/688] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:42.671758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:42.672389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.672566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:42.677673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:42.685468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:42.713279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 342/689] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:42.715318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.797582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3ba02-62e6-43a4-bf90-b42a6ea1e039 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.866189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.867419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.868935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.869451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.871215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=07600562586a47a696ec87ef60838559) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:42.872604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:42.876466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:42.894822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 348/690] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:42 2022] GET /identity/v3/role_assignments?user.id=07600562586a47a696ec87ef60838559&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:42.897436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:42.903533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52232-0555-4bd6-b437-c42c1b5b0545 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:42.943949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:42.977312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:42.978471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:42.979008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:42.979686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:42.980900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:42.981475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:42.983367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:42.995429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 343/691] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:43.002029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.125958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a5c1-a887-4755-a28f-d9d874ca2504 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.187785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.189037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.191376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.191996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.193564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:43.210996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.214849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:43.249071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:43.251254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:43.253225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:43.258534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 349/692] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:43.259521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.290143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab8e7-ae7b-4b5f-85ae-794633915027 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.360227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.362905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.363297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.363623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.376489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:43.380652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.384264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.401550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7401e87-f83c-4eba-a2ff-1010f04ae1ec None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-867e3907-17bf-4a15-b756-68b9a96140f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:43.424223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:43.426365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:43.428071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:43.428071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:43.428071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:43.428071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:43.428071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 344/693] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:43.429653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c04da-02af-4c0e-b209-1aea8d373e06 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.430106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c04da-02af-4c0e-b209-1aea8d373e06 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.430536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c04da-02af-4c0e-b209-1aea8d373e06 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.432393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 345/694] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:43.437064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c04da-02af-4c0e-b209-1aea8d373e06 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.477325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.480339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.481188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.481968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.485512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:43.486584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.489606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4caad0-7960-4358-92c5-51357f6c9c2e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.519966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 350/695] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:43.522136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.522708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.523521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.545891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:43.547375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:43.594233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c04da-02af-4c0e-b209-1aea8d373e06 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.609664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:43.651983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.653178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.653652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.654140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.656111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:43.656945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.659750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.669738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 351/696] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:43.670807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.697363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 346/697] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:43.698727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.812225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36cec-e0d2-4a35-a468-561e21e83e33 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.836763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:43.851746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.856834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.856834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.856834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.866008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.867917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.880541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadf634-cdea-444b-a1d3-be65219c43f9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:43.889657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:43.890683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:43.891582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:43.891582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:43.891582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:43.891582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:43.898461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 347/698] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:43.899705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:43.929038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:43.941535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:43.942329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:43.944076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:43.944076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:43.944076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=444436f77ef74d9fb825aafa19ccc146) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:43.944340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:43.946063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:43.953364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 352/699] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/role_assignments?user.id=444436f77ef74d9fb825aafa19ccc146&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:43.954459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:44.024653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a062-b5a4-4932-a41f-ba911c2acac7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:44.066507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:44.067717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.068230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.068712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.077772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:44.079932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:44.086557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b0ce4-b940-437e-afe8-79e571ea6a3e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:44.097825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:44.098887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:44.099865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:44.102707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 348/700] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:44.111502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:44.134504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:44.135369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.135718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.136111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.137046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:44.137425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:44.139040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:44.153531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 353/701] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:44.154134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:44.164657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:44.262629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:44.278573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56edd80-efc8-4c08-8887-a9b26f8670da None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:44.330159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:44.330871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.331184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.331451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.332298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:44.332545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ada211-322c-4098-8dc9-ff341e21e60c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:44.332992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:44.334044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:44.356639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 349/702] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:15:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:44.367288 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:44.371208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:44.371893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.372174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.372447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.373169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:44.374677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fac53e-3242-4c23-b855-b38efa35f3a9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:44.491246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 354/703] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:15:44 2022] POST /identity/v3/users => generated 254 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:44.561872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cda0f-e298-4c0b-bc63-0a36f1273c80 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:44.635501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:44.637269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.637767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.638296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.661267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:44.784107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d075-f8e3-448b-b5a8-f92dfe331d13 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.784790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d075-f8e3-448b-b5a8-f92dfe331d13 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.785367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532d075-f8e3-448b-b5a8-f92dfe331d13 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.788607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 355/704] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:44.793576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:44.793854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:44.794137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:44.806972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:44.809011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:44.834604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:44.836038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:44.938805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:44.960887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 350/705] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:44.971759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 356/706] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:44.972828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.028190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.039107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:45.107761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d137a-2120-463e-ab61-541639f64f5d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.145747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.146571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.146854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.147119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.159210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.160680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:45.179702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:45.181433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:45.183168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:45.183168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:45.183168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:45.183168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:45.185666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 351/707] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:45.187065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.221154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4988-8b39-4361-bdd7-1c55c3cb0a13 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.288260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.290181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.291009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.291816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.292130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:45.295109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.298324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.303281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611c0b-219d-41c9-83ec-8c6cb7cd80f9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:45.342959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:45.345378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:45.347088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 357/708] 10.222.0.48 () {56 vars in 1129 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:45.354232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.354232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.354232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.354232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.355928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:45.355928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.362326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.362683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.393896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 352/709] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:45.396682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.473821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:45.583773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae8d40-53bb-41da-b1b2-03ec5bd6a9fd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.615039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1221e5-c633-4823-a670-38ccfea2d716 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.660404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.661080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.661350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.661627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.662380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:45.662691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.664091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.677180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 358/710] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:45.684016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.685844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.686685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.686964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.687238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.703797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.705294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:45.720612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:45.722444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:45.723937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 353/711] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:45.727694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.750358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:45.857647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad832-1468-4219-a57a-e85d0727ec90 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:45.904007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:45.904580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:45.904842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:45.905114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:45.905980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:45.906314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:45.907780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:45.939010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 354/712] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:45.942131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:45.973006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c7510-1dfd-4332-a1d6-03e2c12946cb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:46.006582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:46.013082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:46.013760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:46.014045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:46.014355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:46.015224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:46.015538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:46.017008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645ca6d-c2d8-4fed-b1c0-45cd65bcd4a6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:46.023955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 359/713] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:46.082357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:46.117557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711b372-1382-4fb2-9dd5-024aa08c8145 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:46.157527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:46.158207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:46.158499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:46.158774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:46.159642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:46.159961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:46.161722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac576e64-c626-43c4-875c-36c379ea33b8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:46.169252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 355/714] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:45 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:46.521600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3492-d074-4f82-ad11-acffab6ae001 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:46.522149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3492-d074-4f82-ad11-acffab6ae001 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:46.522430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3492-d074-4f82-ad11-acffab6ae001 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:46.524301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 360/715] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:46.530658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:46.531107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:46.531485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:46.551115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:46.552534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:46.638355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 356/716] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:46.656633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3492-d074-4f82-ad11-acffab6ae001 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:46.778859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3492-d074-4f82-ad11-acffab6ae001 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:46.815672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:46.816482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:46.816763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:46.817035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] PATH_INFO: `/v3/services/network` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:46.817692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:46.819132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:46.833222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:46.834534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:46.835551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:46.838178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 361/717] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:46 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:46.845026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:46.922758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:46.962518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2281-2cda-4dd8-aae9-9a858fea6ef2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:47.001843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:47.002852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.003296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.003743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.004660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] RBAC: Adding query filter params (name=network) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:47.005114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:47.006771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67923e8-5fb8-427e-8835-a0ae1ac0be84 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:47.012501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 357/718] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:15:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:47.018513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:47.084515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:47.147315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b93cf2-44c5-4af3-bb01-3a6afe2804ce None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:47.208457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:47.209571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.210064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.210562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.211488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] RBAC: Adding query filter params (type=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:47.212011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:47.213615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f144f5-1e91-4297-a458-5727f516d41d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:47.222340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 362/719] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:15:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:47.303757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bc831-215a-4184-9005-b5eb565258e4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.304129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bc831-215a-4184-9005-b5eb565258e4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.304312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bc831-215a-4184-9005-b5eb565258e4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.306214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 358/720] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:47.312209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dcd0f-6527-454f-b9c1-35e5efeb9293 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.312860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dcd0f-6527-454f-b9c1-35e5efeb9293 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.313402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dcd0f-6527-454f-b9c1-35e5efeb9293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.344937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-679dcd0f-6527-454f-b9c1-35e5efeb9293 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:47.346592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-679dcd0f-6527-454f-b9c1-35e5efeb9293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:47.444956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 363/721] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:47.488207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bc831-215a-4184-9005-b5eb565258e4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:47.521266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd6cb-00cc-4f28-b66d-acdcd48ba227 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.521606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd6cb-00cc-4f28-b66d-acdcd48ba227 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.521818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd6cb-00cc-4f28-b66d-acdcd48ba227 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.523729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 364/722] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:15:47.529465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.529811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.530100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.550444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:47.551672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:47.585544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:47.636126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 365/723] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:47.652202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bc831-215a-4184-9005-b5eb565258e4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:47.657926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:47.710573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:47.721942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:47.726615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.726615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.726615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.751519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:47.763664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:47.764737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:47.765589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 359/724] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:47.770980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:47.876196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973cab6-7253-4716-88e3-900418e8ace1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:47.955870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:47.957001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:47.957478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:47.958293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:47.978847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:47.986062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:47.987161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:47.987983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:47.988624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 366/725] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:47.998985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.007253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58520f76-ecee-4f2e-b4b1-98ce7c96f2d9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.066576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:48.085112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.085795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.086075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.086390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:48.087149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:48.087455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:48.088848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:48.107137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 360/726] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:48.114220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.213480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad9a9-4f29-456b-92aa-282cc2451aa1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.228435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:48.287031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.287707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.287987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.288262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:48.289010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:48.289314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:48.290733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:48.306455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 367/727] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:48.312910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.321509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddd553-504e-4e65-bf1d-f308e190c5f0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.366083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.366935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.367217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.367489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:48.382538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:48.384038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:48.404920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:48.405914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:48.406873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:48.406873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:48.406873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:48.406873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:48.406873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 361/728] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:48.413095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.535910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:48.584599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7082235-d951-4b31-a2c0-12d1a22ac9fa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.663970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.665212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.665698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.666184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:48.670637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7919e-c6b6-4b2d-b924-04ee6fe08676 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.690852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:48.700767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:48.703501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:48.747490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.748588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.748974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.749389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:48.750790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:48.751406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:48.754080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:48.762329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:48.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:48.764223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:48.764223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:48.764223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:48.764223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:48.781419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 368/729] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:48.784574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.799644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 362/730] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:48.807289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:48.943010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc0a-9dbb-47d8-af32-ba58814e1113 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:48.997754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:48.998919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:48.999471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:48.999976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.001004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:49.001585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.003290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.021575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.030094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 369/731] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:49.034317 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.064232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6597d6-d5ea-4c1c-965e-f689f5b229f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.156888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2c2e9-0422-43a8-b558-978e7bf14811 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.184684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.185498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.185875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.186377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.194839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.196130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.196640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.196940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.197296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.197498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.206593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.208205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-40277f2c-370c-4800-8ee0-96810fe5c6b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:49.213387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:49.216350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:49.219924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:49.221796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 363/732] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:49.228399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaab12c-b347-4f4f-a980-d6fa5379aaf0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.228399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaab12c-b347-4f4f-a980-d6fa5379aaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.228399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaab12c-b347-4f4f-a980-d6fa5379aaf0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:49.229557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:49.233088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:49.242639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 370/733] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:15:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:49.244661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 364/734] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:49.244823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.245045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.245045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.245045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.258001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:49.259517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:49.294688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.309020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.346570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 365/735] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 09:15:49.347929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.381571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868777f-e601-4c22-9fb6-a4098a03c7be None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.428976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.429664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.429938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.430223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.431091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:49.431400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.434216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.461606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 371/736] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:15:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:49.462694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.500778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781372e-47ba-4044-9098-2c43189d1399 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.541843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.543095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.543391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.543635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.544491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:49.544796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.546298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e95da-8c4d-40d2-bae2-f4c5fec42f8e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.571565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 366/737] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:15:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:49.572759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaba01-bb77-4e86-8533-d8f63e1d1675 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.573083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaba01-bb77-4e86-8533-d8f63e1d1675 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.573352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaba01-bb77-4e86-8533-d8f63e1d1675 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.575543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 367/738] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:49.576539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaba01-bb77-4e86-8533-d8f63e1d1675 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.594482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.621453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd18ca6-9b52-4d06-8d67-69db4e1af310 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.644892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.664609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.665318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.665647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.665902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.666652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.668662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.706384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaba01-bb77-4e86-8533-d8f63e1d1675 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.743991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.744644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.744923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.745193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:49.786280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 372/739] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:15:49 2022] POST /identity/v3/services => generated 251 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:49.786280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:49.864323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:49.866988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ad10b-e18b-48be-ba09-6872b7b5d401 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:49.893517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:49.936280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d205255-fdc2-4a40-8a1c-1b3659cada9e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:49.985100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:49.985994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:49.986376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:49.986602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.038906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 368/740] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:50.040607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56c4d-49dd-441d-9fcd-8772c35a6733 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.041068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56c4d-49dd-441d-9fcd-8772c35a6733 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.041504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56c4d-49dd-441d-9fcd-8772c35a6733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.055975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:50.061104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-19c56c4d-49dd-441d-9fcd-8772c35a6733 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:50.062489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-19c56c4d-49dd-441d-9fcd-8772c35a6733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:50.099240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:50.100763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152023a2-45a9-4eb3-b768-c74832894ab6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:50.169248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 369/741] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:50.171989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b06b-3edb-4796-b3d6-c98d2609da90 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b06b-3edb-4796-b3d6-c98d2609da90 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.172540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b06b-3edb-4796-b3d6-c98d2609da90 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.175474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 370/742] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:50.176919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b06b-3edb-4796-b3d6-c98d2609da90 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.241800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 373/743] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:15:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 09:15:50.243597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.244119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.244585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.245093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:50.264312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:50.265527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:50.328361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b06b-3edb-4796-b3d6-c98d2609da90 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:50.366250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 374/744] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 09:15:50.366250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.368800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:50.407595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:50.409013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.409585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.409940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.422259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:50.423877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:50.446958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:50.454111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 371/745] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:50.455637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.508977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df79e87-16ad-4bd1-8028-feb9c5e2699a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:50.576461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:50.577719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.578230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.578733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.583935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:50.592299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a875d-bd9d-4a32-aff9-84eb2647ef3b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:50.600439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:50.615621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:50.617275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:50.618474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 375/746] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:50.621273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.642709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:50.644074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.644480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.644947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.659042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:50.660741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:50.684087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:50.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 372/747] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:50.688401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.747483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:50.823063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031625d4-10fb-4b4c-b99f-780fb68964c7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:50.834564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418b570-a0d4-46ae-b5ee-9202644d67c9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:50.876405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:50.877307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.877717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.878250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.879100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:50.879610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:50.881520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:50.888029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:50.889130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:50.889628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:50.890117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:50.891477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:50.894074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:50.902575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 373/748] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:50.902575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:50.916120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:51.001801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 376/749] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:15:50 2022] POST /identity/v3/services => generated 240 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:15:51.003332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.084745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f28d5-603e-4492-94ed-619bc75e818e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.160294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.161653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.162284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.162860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.164364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:51.165029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.168814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.190891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 374/750] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:51.192569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.236850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7511-c07f-4752-9d03-38e48f45e3c7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.282274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:51.312042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.313207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.313546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.314009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.315771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:51.315771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.318252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.334496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:51.344583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 377/751] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:51.346306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.400861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e53ddb-83d0-48e8-940c-204ee3617b48 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.471436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.473263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.474013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.474749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.488524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.491681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:51.524891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:51.527032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:51.529226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:51.533465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 375/752] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:51.535402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.554630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee5235-d0b9-4f3a-9228-9c84ddf5725c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.623157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.624609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.625173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.625744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.647982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.651962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:51.688778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:51.690573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:51.692360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:51.692360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:51.692360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:51.692360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:51.692360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 378/753] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:51.693261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.760454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043e3f5-e9fc-4438-8261-3ce758687344 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.761253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:51.809604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:51.831805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.833649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.834526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.835214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.849036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.852706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:51.882685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:51.884713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:51.887202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:51.894592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 376/754] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:51.894592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:51.898878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92251-b615-40a1-82d2-d1eb0a348d74 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:51.963364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:51.964931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:51.965667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:51.966569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:51.968973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:51.969848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:51.973235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:51.999680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 379/755] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:52.002166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:52.075289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67955730-480f-4a81-85a6-41782f70dbf9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:52.139109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:52.140938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:52.140938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:52.141260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:52.142471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:52.142999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:52.145413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:52.168179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 377/756] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:52.169761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:52.196500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:52.207018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddca21-8bf5-493e-99d9-2640824ac59b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:52.257612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:52.274877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:52.276413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:52.277718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:52.278484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:52.280198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:52.281001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:52.284023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:52.318653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 380/757] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:52.320921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:52.356556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1ffe8-0dfd-45d4-9e8c-584d2554ba93 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:52.419727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:52.420755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:52.421638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:52.421802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:52.423306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=444436f77ef74d9fb825aafa19ccc146) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:52.423838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:52.426401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:52.437893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 378/758] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:52 2022] GET /identity/v3/role_assignments?user.id=444436f77ef74d9fb825aafa19ccc146&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:52.439319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:52.522137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b622e-f835-4ad2-8030-710ea10248cf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:52.653256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:52.654490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:52.654980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:52.655476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:52.685117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:52.687892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:52.718231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:52.720090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:52.721982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 381/759] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:15:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:52.731902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:52.758562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae410a2-e250-41ba-8407-068e1ce97118 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:52.781298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:52.787272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:52.832395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:52.834010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:52.834804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:52.835569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:52.837527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=86d945bffa964372bec61684a4a1e95a) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:52.838411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:52.842069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:52.855629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 379/760] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:15:52 2022] GET /identity/v3/role_assignments?user.id=86d945bffa964372bec61684a4a1e95a&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:52.995699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5112f0-358b-4287-8f55-57ec7b144e14 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:53.088112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:53.089767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:53.090285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:53.090770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:53.092391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:53.092944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:53.095718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ebdce-aee0-4e34-bc92-4dabd0be73c9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:53.152734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 382/761] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:15:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:53.161596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:53.194508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c70d-a1ad-4f8e-93c0-1825e2e0b3b6 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:53.194775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c70d-a1ad-4f8e-93c0-1825e2e0b3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:53.195068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8c70d-a1ad-4f8e-93c0-1825e2e0b3b6 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:53.197090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 383/762] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:15:53.206105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:53.206621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:53.207119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:53.243937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:53.246502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:53.394861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22194b6-2a9a-4010-a803-dc551866b39c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:53.431652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 384/763] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:15:53.469814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:53.471143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:53.471623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:53.472101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:53.482088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:53.487296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:53.519049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:53.526999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:53.582419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:53.584123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:53.585902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:53.585902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:53.585902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:53.585902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:53.585902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 380/764] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:53.591380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:53.685758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:53.778274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e908b-0999-4b0d-83c4-9eb18dcbf6ad None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:53.842628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:53.843945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:53.844444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:53.844926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:53.846166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:53.848812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:53.873266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:53.875166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:53.876357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 385/765] 10.222.0.48 () {56 vars in 1129 bytes} [Fri May 27 09:15:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:53.882339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:53.934648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780de71d-2b85-451e-adc9-afa0659cd569 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.060548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.061142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.061423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.061699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.062462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:54.062776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.064177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.092832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 381/766] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:54.095641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.204016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:54.281274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3868cb-ed2d-42af-ae2f-23656d9817cf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.350339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:54.358665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe48b-3386-454f-99dc-c1217dac5d7d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.368935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.370273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.370743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.371198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.372244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:54.372886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.383162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.392726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 386/767] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:15:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:54.399718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.400552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.400904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.401182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.402746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.421073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.423027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:54.455351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:54.456326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:54.457257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:54.458089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 382/768] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:54.467907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.557635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce11b-2278-4922-9431-7030963742e7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.605322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:54.610977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.611983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.612488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.613090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.613977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:54.614495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.623333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 387/769] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:15:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:54.630156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.785670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee037-0bac-4a1e-be0b-326b9de4440b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.806340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9800e3-cd60-4c14-b2db-653b542f0042 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:54.824740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.825778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.826306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.826783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.827831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:54.828351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.830035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.848612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 383/770] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:54.850278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.855271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:54.859661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:54.860389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:54.860539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:54.860856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:54.861584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] RBAC: Adding query filter params (interface=public, service_id=cbabb2819d60443480c7b2d7e0b5770c, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:54.861893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:54.867622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:54.886271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 388/771] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:15:54 2022] GET /identity/v3/endpoints?service_id=cbabb2819d60443480c7b2d7e0b5770c&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:54.887239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:54.964736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddc71c-4eb1-4f54-b09f-ac87d07f16de None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:55.005123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:55.006081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.006405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.006685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.009079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=55612ce26632413ebd0dfd5fc081b656, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:55.010695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88b64c-225e-44ca-8c40-c8d3d90d5d70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:55.035352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:55.081782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1740e-a6a4-4cc7-901d-6d44d4afff3d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:55.096085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 384/772] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:54 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:55.131990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:55.132593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.132871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.133143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.133862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:55.135338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8131a-04aa-44c5-9aac-ebbf07696a48 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:55.141839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 389/773] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:15:54 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:55.337313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0c2c1-3349-4133-ba1f-2d3a60ec5776 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.338414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0c2c1-3349-4133-ba1f-2d3a60ec5776 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.339253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0c2c1-3349-4133-ba1f-2d3a60ec5776 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.342612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 385/774] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:55.350448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.350836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.351064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.378841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:55.380148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:55.466786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 390/775] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:15:55.512156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0c2c1-3349-4133-ba1f-2d3a60ec5776 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:55.615730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:55.679361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0c2c1-3349-4133-ba1f-2d3a60ec5776 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:55.732279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:55.733023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.733304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.733581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] PATH_INFO: `/v3/services/network` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.734347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:55.735826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f74227da-f1e9-40d9-9d26-1a8105a75f8f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:55.750994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:55.752066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:55.752918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 386/776] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:55 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:55.758952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:55.864218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:55.933482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d0703-dfad-469e-944b-eb13e9afe766 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:55.960435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de956845-4efd-439e-82e8-1cd7603e41bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.960766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de956845-4efd-439e-82e8-1cd7603e41bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.961016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de956845-4efd-439e-82e8-1cd7603e41bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.963620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 387/777] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:55.973104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.973599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.974085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.982023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:55.983099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:55.983323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:55.983598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:55.985278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] RBAC: Adding query filter params (name=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:55.985619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:55.987154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:55.992919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 391/778] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:15:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:55.999801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.027946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:56.029160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:56.132806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 388/779] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:15:56.143775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d840e-f290-4bbc-b6da-e62feff7eefc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:56.184545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:56.191082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.253677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:56.254403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.254690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.254967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.255717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] RBAC: Adding query filter params (type=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:56.256026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:56.257463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:56.273124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 392/780] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:15:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:56.294451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.358802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:56.448035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d89e3-980d-4a8e-bf3b-f9b78290a4f0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:56.471364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df2f6-385b-4273-b5e9-97de1345473d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:56.508768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:56.509553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.509935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.510361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.511309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] RBAC: Adding query filter params (interface=public, service_id=f089ba5b805745bf96695803034b35f8, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:56.511719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:56.513276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c47b94-b0e5-451d-b9d1-651b8fdf24c5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:56.515910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:56.518067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.518810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.519499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.520014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 393/781] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/endpoints?service_id=f089ba5b805745bf96695803034b35f8&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:56.525870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de133-e296-4447-8e67-0a8f5a17a37b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.526184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de133-e296-4447-8e67-0a8f5a17a37b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.526563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de133-e296-4447-8e67-0a8f5a17a37b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.528410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 394/782] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:56.529280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0de133-e296-4447-8e67-0a8f5a17a37b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.544127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fc89d513-e10b-4dba-b654-b2b5cc2b287b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:56.560986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:56.562941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:15:56.564662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:56.572136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 389/783] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:56.575038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.575785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.576592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.606625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:56.609580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:56.612058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:56.643961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0de133-e296-4447-8e67-0a8f5a17a37b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:56.684223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:56.684995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.685333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.685671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.686507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:56.687976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:56.702550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 395/784] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:56.704139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.753309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 390/785] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:15:56.791419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.837550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e405bd-f06f-4239-af9d-5e239854e5ab None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:56.851678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:56.884901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:56.885914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:56.885914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:56.886168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:56.887089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:56.887458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:56.889408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:56.902638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 396/786] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:56.909170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:56.940153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:57.023653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319e83-3309-4f4b-a5cd-bf4f32e5e748 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.046431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aeb055-5dcf-4717-ab0e-3592400e3150 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.061506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.062458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.062802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.063135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.076598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.078396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:57.097846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:57.099660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:57.101339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:57.101339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:57.101339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:57.101339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:57.101339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 397/787] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:57.106663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.128761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.130690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.131538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.132364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.136293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:57.155264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:57.172188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:57.174011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:57.175700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:57.181327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 391/788] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:57.194067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.236362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526550-ea1b-4f66-a156-72138596e12d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.286027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.286850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.287134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.287406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.288154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:57.288461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.289885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.305476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 398/789] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:57.311962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.355768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35a2a5-1fb7-4719-8fd6-cdb3fb2a4635 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.367196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:57.396935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.398592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.399135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.399627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.400691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:57.401217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.403503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.416535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:57.424147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 392/790] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:57.430377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.437828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9965a-8404-46fb-b41b-b4c78597cf39 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.480401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.481636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.482117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.482796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.491430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.493320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:57.511930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:57.513156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:15:57.514397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:57.516891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 399/791] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:57.523080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.642992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdeb8f-2636-4e33-bd00-518caf8aa937 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.683337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.684404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.684871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.685348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.686556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:57.687069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.688721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.711403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 400/792] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:15:57.717291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.722728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:57.771917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ef621-068e-439e-ba3b-f6710665a2e0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.857396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.858571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.859058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.859511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.878815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.882009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.884697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00503d-4b08-49df-861a-0361c828e8b6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:57.912652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:57.913612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:57.914570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:57.914570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:57.914570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:57.914570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:57.916650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 393/793] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:15:57.933126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:57.944022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:57.944847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:57.945132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:57.945668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:57.960041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:57.961531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:57.989982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:58.020268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:58.021249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:58.022178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:58.022178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:58.022178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:58.022178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:58.035057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 401/794] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:58.036690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.210282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4ecf9-c131-44a3-bc7a-1139c25d9b23 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:58.250745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:58.252561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:58.253255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.253542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.253806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.254789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:58.255097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:58.256522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:58.289038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 394/795] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:58.295369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.329636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a43fb0-12ff-4a3a-9b23-f0fedfcc27a6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:58.371122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:58.371811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.372100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.372366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.373476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:58.373910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:58.375493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637c03-8bbe-4029-9011-61db5da94962 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:58.390320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 402/796] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:58.391984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f15dd-dd04-494f-b526-59d469dc1675 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.392377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f15dd-dd04-494f-b526-59d469dc1675 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.392755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f15dd-dd04-494f-b526-59d469dc1675 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.394568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 403/797] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:58.400657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.400990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.402605 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.420488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:58.422052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:58.510787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 404/798] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:15:58.511889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.525234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:58.567322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6fee7-ad30-43bf-9f07-4361e9ae859a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:58.609002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:58.610111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.610624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.611046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.619941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:58.622224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:58.626597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:15:58.640497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:58.641621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:15:58.642698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:58.645102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 395/799] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:58.646383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.732682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ce9ca-7b2c-445f-9c9b-873c5891d659 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:58.778483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:58.779563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.779999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.780417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.789092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:58.790980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:15:58.807554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:58.809192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:15:58.810091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:15:58.817158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 405/800] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:58.818596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.844206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b1854-5962-4953-8c97-ec75a2e59f13 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:58.884352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:58.885494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:58.885998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:58.886482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:58.898465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:15:58.907156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:58.908159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:15:58.909004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:58.911934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 396/801] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:58.913060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:58.928699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.023153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb60a0-9f86-4306-ae20-fa9b4fd9184b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.028172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.077124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.078054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.078407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.079166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.079482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:59.079792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.081255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.094848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6536-7b46-4624-8b45-d9776a13b3ba None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.111977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 406/802] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:59.113152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:59.151171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.152141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.152698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.153104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.154066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:59.154619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.156455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee337000-7f58-4b38-922d-41d9c3f04d5a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.172836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 397/803] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:59.174970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603e03-7d87-4e32-9b97-74027d19fc16 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.175469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603e03-7d87-4e32-9b97-74027d19fc16 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.175932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b603e03-7d87-4e32-9b97-74027d19fc16 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.182380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 398/804] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:15:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:15:59.182380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b603e03-7d87-4e32-9b97-74027d19fc16 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:59.280505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447e785-718e-43fd-9ffc-5fa9fa0f7727 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.292884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.325206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.326189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.326667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.326984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.327879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:59.328112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.329724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.340857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 407/805] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:15:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:59.342274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:59.378105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.382965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b603e03-7d87-4e32-9b97-74027d19fc16 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.449506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.451619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.452101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.452594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.475945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.479224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.486704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cbfac-ff45-4d9e-8fbd-99049543774c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6245ce60-4380-4db3-ade4-43cb3c8f7c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:15:59.517540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:15:59.518836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:15:59.519845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:15:59.519845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:15:59.519845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:15:59.519845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:15:59.519845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 399/806] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:15:59.521875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.522557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.522851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.541996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.545624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.546261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.546773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/86d945bffa964372bec61684a4a1e95a/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.553198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:15:59.555257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:15:59.571260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=86d945bffa964372bec61684a4a1e95a, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.574261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.649192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 408/807] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:15:59 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/86d945bffa964372bec61684a4a1e95a/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:15:59.650782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:59.660956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 400/808] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:15:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:15:59.666082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:15:59.769128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.838161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:15:59.883676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e65c-d1d6-4f23-9567-338538f523d3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.887731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcaba8-22bb-4c1a-b5ab-944a04539aba None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:15:59.951732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.952685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.953136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.953290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:15:59.953610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.954467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:15:59.954907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:15:59.955066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:15:59.955426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:15:59.955688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.957057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.970680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:15:59.972660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:15:59.976900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 401/809] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:15:59.978312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3e59cb89-f361-4d2b-8f70-43e9cc18b0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:00.005945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:00.007681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:00.009467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:00.009467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:00.009467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:00.009467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:00.009467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 409/810] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:00.012226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b4e7-d33a-4369-94f2-4c5dc97fd0d2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.012715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b4e7-d33a-4369-94f2-4c5dc97fd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.013191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637b4e7-d33a-4369-94f2-4c5dc97fd0d2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.016843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 410/811] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:00.018931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637b4e7-d33a-4369-94f2-4c5dc97fd0d2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.132422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d941b9-f799-474b-9660-98253b83ebbe None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.135291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:00.182108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.184470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.185442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.187414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.188735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.191357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.203393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637b4e7-d33a-4369-94f2-4c5dc97fd0d2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:00.215103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:00.216191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 09:16:00.217137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:00.219340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 402/812] 10.222.0.48 () {56 vars in 1129 bytes} [Fri May 27 09:15:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:00.219340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.246886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:00.259002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.261487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.262231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.262911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.278032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.282006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5fe8806e-d6f9-4085-8e6a-6a1a33ff6672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:00.310180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:00.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 411/813] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.314190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.315647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.339992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:00.341665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:00.404471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee6e0-0072-4e73-b227-1597cfd47ff1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.434800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 412/814] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:00.435526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.471604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.473244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.474050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.474872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.476563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:00.477104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.479842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.501102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 403/815] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:00.501102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.530763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:00.588832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e48bf-5163-4c3b-87ae-f87ece7d6285 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.623424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:00.625108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5e1f4-9bec-4f1c-b64c-092c58eb071a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.649101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.649721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.650000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.650304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.651022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:00.651328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.652868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.657700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 413/816] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:00.658579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.686690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.687840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.688347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.688852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.690458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:00.691031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.693804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.715084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 404/817] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:00.716377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.815783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:00.820969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415899-bd0f-4bde-be8f-bd8fa9c9ac59 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.829294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ea177-428e-43cb-b0f6-dda214963d52 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:00.866721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.867529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.867810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.868083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.877030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.878567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.886341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:00.888053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:00.888751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:00.889475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] PATH_INFO: `/v3/services/network` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:00.890935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:00.893841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:00.896225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:00.898280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:00.900752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:00.900989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 405/818] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:00.900989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:00.908908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:00.912793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:00.916614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 414/819] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:00.920640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:00.966899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:01.040580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13b20a-311b-429b-8dfe-6d7fe72c314f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.041370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff793f-1092-4274-aeee-42f0c1bdba9f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.088368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.090779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.091492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.092196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/444436f77ef74d9fb825aafa19ccc146/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.093689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.094777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.095246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.095712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.096673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=444436f77ef74d9fb825aafa19ccc146, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.096903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:01.097413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.099821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.100301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.109737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 406/820] 10.222.0.48 () {56 vars in 1124 bytes} [Fri May 27 09:16:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:01.109737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:01.130280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:01.161281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 415/821] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:16:00 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/444436f77ef74d9fb825aafa19ccc146/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:16:01.162960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:01.261824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeead8-f23f-4e42-a6fe-369d66218f74 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.322092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.323165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.323693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.324154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.325226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:01.325771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.326911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281299-551f-40b6-8c64-b3c667d48bea None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.328126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.331504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:01.350338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 407/822] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:16:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:01.351334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:01.394555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.395967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.396675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.397374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.399627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] RBAC: Adding query filter params (name=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:01.400866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.404428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.418569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 416/823] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:01.420462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:01.456244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:01.480033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9ad6b-280d-4c64-939a-7a4b605cca09 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.524690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.525544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.525928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.526376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.527779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.529212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.626258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c38242-5c32-46d8-8c7a-919ffccd3e0f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:01.665398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 408/824] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:01.667101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:01.730557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:01.734049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:01.734600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:01.735051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:01.802807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:01.864528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:01.867127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:01.937422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73bfcf-1419-47e7-9986-9fd5844312cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:02.026916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:02.045695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:02.047263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.048053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.048798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.050357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:02.051177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:02.053878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:02.058602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 417/825] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:02.066360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 409/826] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:02.068439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:02.073023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:02.199283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:02.266033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0b309-81b4-4381-8340-94c399397bc8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:02.321105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85359f9-ba27-4c3e-b063-9bc671aec1a9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:02.328975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:02.330547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.331372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.332151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.333832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:02.344138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:02.414562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:02.417016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.417768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.418572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.441904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:02.445710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:02.493041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 410/827] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e001f453-11bc-446c-aebc-654b915d5f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:02.509139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:02.511434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:02.513102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:02.513102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:02.513102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:02.513102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:02.517602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 418/828] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:02.525720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:02.630882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:02.687984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60001c8-4132-461f-8895-4e6b45828931 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.688961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60001c8-4132-461f-8895-4e6b45828931 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.689633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60001c8-4132-461f-8895-4e6b45828931 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.692252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 419/829] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:02.698970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.699365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.699550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.739345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:02.740554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:02.760566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e078035-70b1-4cb1-af10-c1706cca7340 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:02.844234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:02.845938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:02.846784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:02.847540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:02.849136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:02.849775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:02.852451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:02.871742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 420/830] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:02.875183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 411/831] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:02.880696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:02.895772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:02.950859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.027774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d4ba-fdfe-4121-8813-58b80f40e23b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.035031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f4d8f-73b6-408b-8e6b-6b828d0d9e06 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.036838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.090693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.092038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.092745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.093223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.094715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.095142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.095419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.095688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.103863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.106516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.110335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:03.119160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:03.120234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:03.121608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 412/832] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:16:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:03.131672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:03.132637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:03.133504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:03.133930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.136932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 421/833] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:03.143684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.258447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f3398-7c72-4912-8c92-c3108b04270f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.297093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.298146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.298665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.299167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.300129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:03.300664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.302338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.317370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 413/834] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:03.318703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.332358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3bd66-7c2b-4419-a5df-0886bbf5046a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.348865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.366956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.397482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.398620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.399220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.399596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.401185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:03.402324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.405095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.433800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 422/835] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:03.442995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.447575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fec19-578f-4ecd-aa22-0fa86d7e8866 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.502179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.503462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.504103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.504561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.521503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.524200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.551299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce12bd8-e5e4-46d5-82b2-09fe59b35d56 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:03.556470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:03.557388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:03.558365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:03.558365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:03.558365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:03.558365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:03.564918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 414/836] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:03.566173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.591235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.591963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.592248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.592526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.593422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:03.593744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.595280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.608259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 423/837] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:03.680948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.715693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecb80a-0e20-4511-a4e4-77ec6f7f5161 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:03.768391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:03.769292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:03.769589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:03.769924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:03.770706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:03.771015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:03.772450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e94188-a1f2-45de-b92d-b0c1c252485e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:03.794870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 415/838] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:03.801066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:03.842372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:03.983891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249848b-229f-4f78-a27b-de0875e28271 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.045517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.047458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.048276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.048819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.061434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.063600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:04.088953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:04.090127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:04.091451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:04.092519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29ec6-239f-479e-8838-a88d89799cea None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.092962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29ec6-239f-479e-8838-a88d89799cea None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.093050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29ec6-239f-479e-8838-a88d89799cea None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.094574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 424/840] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:16:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:04.094700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 416/840] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:04.100021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.100304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.100579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.101189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.125462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:04.127614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:04.220044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:04.226149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 417/841] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:04.241075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb708a2-4dbb-41b4-8559-363bf323d090 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.249387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.308664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.309955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.310624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.311047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.312271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:04.312748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.314614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:04.345063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 425/842] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:04.346359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.375517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:04.426254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5065c-927f-4bf1-b6d6-f29863e77858 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.474073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.475474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.476171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.476606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.491846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:04.492055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d178dd-071a-4d68-839d-97c607324591 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:04.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:04.503882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:04.504859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:04.507175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 418/843] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:04.514164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.558592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.559580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.559865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.560140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.590469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:04.655718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.656388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f501-e4c0-4cd1-8778-b3baa66b591c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.657167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:04.691634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:04.704807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.705885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.706455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.706943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.707953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:04.708720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.710524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:04.726739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 419/844] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:04.729016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.788957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 426/845] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:04.795206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:04.857050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:04.904550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c231c-3aef-4a79-8fe9-481112ca4e64 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.924315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bebc7-32a3-4fbf-a752-51816a70a962 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:04.965827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.966791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.967062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.967376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.969321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:04.971095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:04.971857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:04.972705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:04.980947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.982544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:04.994923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:04.998528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:05.004362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:05.005612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:05.007354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:05.007354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:05.007354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:05.007354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:05.010271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 427/846] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:05.011324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:05.031629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:05.032640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:05.033548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:05.033548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:05.033548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:05.033548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:05.040279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 420/847] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:05.042941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.089851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.124736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.129194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641733-743e-4980-bbf4-6549aa6dd95c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.159511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171d639-7225-4c95-8a11-a4fcd751dcc7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.170793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.171792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.172462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.172716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.173486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:05.173792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.175460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b8720-4b6d-4d88-a512-141cff8484f7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.190144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 428/848] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:05.191725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380360d-fd9d-43b4-820c-dd9d227341fb None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.192008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380360d-fd9d-43b4-820c-dd9d227341fb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.192288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380360d-fd9d-43b4-820c-dd9d227341fb None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.193809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 429/849] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:05.198027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380360d-fd9d-43b4-820c-dd9d227341fb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.210968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.211649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.211979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.212253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.213175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:05.213484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.215165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4afb4-797a-4084-ac0c-78fad5b9340f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.234067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 421/850] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:05.234067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.234067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.234067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.252968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:05.254413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:05.331101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380360d-fd9d-43b4-820c-dd9d227341fb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.342283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 422/851] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:05.342283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.371685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.372766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.373130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.373503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.381993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.383289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.384839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:05.402015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:05.403812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 430/852] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:05.405473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.434341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.472686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53391-ca24-424a-ab00-145aab7f34c4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.513375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.514229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.514501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.514773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.522838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.524328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:05.539935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:05.541776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 09:16:05.544536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:05.546527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 423/853] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:05.547792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.564284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a11a-ca35-4e1d-b21b-855d4144b117 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.631403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.633247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.633975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.634903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.636176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.638620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:05.661254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:05.663047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:05.664739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 431/854] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:05.668107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.676506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.699581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388916e6-fb91-4be6-a433-a0b357831423 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.773827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.774901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.775344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.775784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.777632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:05.778131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.780600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.787904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:05.808055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 424/855] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:05.808055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.827113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff090714-c45c-4e48-bb77-72b1d8511e5e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:05.886047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:05.887246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:05.887695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:05.888161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:05.889508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:05.890080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:05.892664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:05.935447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 432/856] 10.222.0.48 () {56 vars in 1135 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:05.935447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:05.996737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb128e-ef6b-49ee-b931-2ab4a7ca8df7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.057962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.058964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.059430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.059900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.061225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:06.061732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.064291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.072172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 425/857] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:06.074095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:06.106988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:06.129401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356797-b327-4871-9fbc-04e0b3513448 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.179743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.180460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.180728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.181002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.181867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=86d945bffa964372bec61684a4a1e95a) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:06.182175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.183782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.191861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 433/858] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:05 2022] GET /identity/v3/role_assignments?user.id=86d945bffa964372bec61684a4a1e95a&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:06.192832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:06.203801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:06.237846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7c9f-7fe7-4721-b25a-6923eaa24f4c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.295212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.295454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.296697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.296697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.354895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243da571-2c88-4774-8e24-527e1324ca63 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.395924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.397799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.402995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.403685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.403969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.404244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.404960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:06.405318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.406756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def87b1f-add3-4ddd-a446-5ace6d45c0a4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.413236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 434/859] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:06.418255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebee33-d438-465d-8bb6-92dc8cd79865 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.418255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebee33-d438-465d-8bb6-92dc8cd79865 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.418255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebee33-d438-465d-8bb6-92dc8cd79865 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.418255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 435/860] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:06.426550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.427210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.427834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.428706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:06.472679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:06.476635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:06.567513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 426/861] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:06.579127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:06.605406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:06.612608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 436/862] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:06.659995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:06.766693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17ad2-7b5b-40fe-92e7-5f5728b5260d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.794930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:06.840316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.842386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.843217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.844032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.867093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.870405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.872133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031e3c-5fe1-4a75-80c0-9047402c96a5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:06.908751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:06.910733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:06.912642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:06.912642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:06.912642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:06.912642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:06.916991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 427/863] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:06.925278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:06.944203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:06.945901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:06.946436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:06.946927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:06.948832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:06.952193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:06.980881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:06.982828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:06.991768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 437/864] 10.222.0.48 () {56 vars in 1132 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:06.998876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:07.127770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:07.135272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f3cc-09af-41f1-9da2-b5853058c383 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:07.202134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:07.203211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:07.203658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:07.204101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:07.205316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:07.205768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:07.207064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193c3ab-3a26-4934-9504-9c1bac9171fa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:07.207968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:07.233781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 428/865] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:07.239846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:07.276467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:07.277344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:07.277831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:07.278361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:07.279824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:07.280424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:07.283063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:07.298708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:07.299862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 438/866] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:07.301673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:07.432359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6502c-bfd3-4f88-b82b-80da47d7a19b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:07.437819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17387ac-7d19-4293-b0f5-6905e7ffcfd4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:07.466795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:07.476412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:07.477097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:07.477413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:07.477701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:07.478466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:07.484243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:07.484243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:07.494246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 439/867] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:07.503032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:07.503855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:07.504134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:07.504407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:07.513343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:07.514912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:07.530869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:07.532706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:07.534503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 429/868] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:07.542550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:07.762832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac96f0-d6b7-4b2d-a3a6-71421e34ffe2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:07.839913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:07.841585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:07.842507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:07.843279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:07.845151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:07.845971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:07.849170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a096a3c-e978-4069-81d4-c6ab52939386 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:07.875102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 440/869] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:07.884868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:08.042827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:08.072278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdfa95-a1b6-4781-ab74-c907e0d511b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:08.144958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:08.146103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:08.146630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:08.147118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:08.148744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=444436f77ef74d9fb825aafa19ccc146) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:08.149296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:08.152113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46975a2c-075c-490f-b825-287e77b0bb60 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:08.164509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 430/870] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:07 2022] GET /identity/v3/role_assignments?user.id=444436f77ef74d9fb825aafa19ccc146&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:08.782735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8cf8-0483-403a-8567-6534d6b07d3a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:08.784368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8cf8-0483-403a-8567-6534d6b07d3a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:08.785185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8cf8-0483-403a-8567-6534d6b07d3a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:08.788675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 441/871] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:08.794303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281bca-ab43-454c-b2e1-1d8fabbe6394 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:08.795052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281bca-ab43-454c-b2e1-1d8fabbe6394 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:08.795052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281bca-ab43-454c-b2e1-1d8fabbe6394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:08.812993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9c281bca-ab43-454c-b2e1-1d8fabbe6394 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:08.814225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9c281bca-ab43-454c-b2e1-1d8fabbe6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:08.938169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 431/872] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:08.981590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8cf8-0483-403a-8567-6534d6b07d3a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:09.089170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:09.179868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8cf8-0483-403a-8567-6534d6b07d3a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:09.214165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547bd4a-ddee-4746-97fc-050460c0247f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:09.214607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547bd4a-ddee-4746-97fc-050460c0247f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:09.215006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547bd4a-ddee-4746-97fc-050460c0247f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:09.217714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 432/873] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:09.226169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:09.226629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:09.227035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:09.236476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:09.237163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:09.237561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:09.237839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:09.238656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:09.240111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:09.250808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:09.252179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:09.317383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 442/874] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:08 2022] POST /identity/v3/services => generated 251 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:09.349506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 433/875] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:09.390870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:09.605865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522bd1a-2d66-43cd-957f-2e2aae217fc5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:09.631769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:09.673498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:09.674940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:09.675395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:09.676040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:09.693576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:09.711869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:09.712892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:09.713806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:09.715596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 443/876] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:09.721660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:09.736196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:09.898660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a80c53-b754-477a-8b6c-70dbce634751 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:09.961873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:09.963022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:09.963502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:09.964582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:09.965975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:09.966560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:09.969256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:09.986502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 434/877] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:09.995543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:10.173774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5100c06-153c-48c1-b35a-090dd2cb3168 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:10.231277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:10.232456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.232943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.233448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.246708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:10.248705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d45c376e-4f62-47a3-8748-dfb8008b0844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:10.270421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:10.271397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:10.272276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:10.272276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:10.272276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:10.272276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:10.274857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 444/878] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:10.287199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:10.396933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:10.449872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b013d-ffe7-4b99-a728-3217e4d6c65f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:10.484388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572db50b-a793-4d2c-9bda-78afdee4caa3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.485042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572db50b-a793-4d2c-9bda-78afdee4caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.485526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572db50b-a793-4d2c-9bda-78afdee4caa3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.487697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 445/879] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:10.490595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:10.491272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.491555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.491830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.492561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:10.492865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:10.496689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:10.501736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.502191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.502191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.510597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 435/880] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:10.518217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:10.522386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:10.525365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:10.554699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:10.632563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 446/881] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:10.656247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:10.687387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871cfbe-8fb6-4bfb-a7fc-161f55c356a1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:10.742835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:10.759274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:10.760657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.760949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.761848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.782059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:10.786013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:10.809406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcbc4a-fd14-42f8-8ab1-ba1d277b2cb6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:10.821950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:10.824142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:10.826486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 436/882] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:16:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:10.838413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:10.874331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:10.875969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:10.876794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:10.877619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:10.879349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:10.882352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:10.969434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 447/883] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:10 2022] POST /identity/v3/services => generated 269 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:11.002786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee36f69-a53f-4258-ade9-e6ba09ed5d46 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:11.059213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:11.060429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.060981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.061470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.062884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:11.063486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:11.065294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7340f-5867-46b9-b66e-54e0adfddae9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:11.088493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 437/884] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:16:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:11.098057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:11.132351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:11.252192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf613-c56a-4388-8469-317c29989750 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:11.296983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:11.298048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.298604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.299118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.356444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482089e-43ea-4244-9de0-7d74f91003fc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.356921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482089e-43ea-4244-9de0-7d74f91003fc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.357388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482089e-43ea-4244-9de0-7d74f91003fc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.360318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 438/885] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:11.368785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.369327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.369746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.389284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:11.391273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:11.407211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:11.410310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:11.422820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:11.466247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:11.550407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 439/886] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:11.574210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 448/887] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 09:16:11.584575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:11.585100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:11.730269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7cde-757d-4910-9b5a-69a49a38916b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:11.772768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c74aa-a02d-49c0-bfbe-1eac5159a1a5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:11.784034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:11.785074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.785363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.785743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.786876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:11.788353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:11.792106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:11.810329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:11.811698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:11.812867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 440/888] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:11.813677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:11.826066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:11.828351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:11.829255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:11.830182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:11.857538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:11.862337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:11.870895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:11.910943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:11.912014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:11.913435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:11.913435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:11.913435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:11.913435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:11.923120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 449/889] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:11.927097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:11.974795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbd54e-8aea-47e6-8f60-6af6a75e0162 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.047936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.048881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.048881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.049168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.049902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.050236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.051795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb2461-ec43-4766-a1e1-3ae769391d67 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.062361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 441/890] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:12.062361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53670-3229-4f56-a425-ae90c070b53a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.062361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53670-3229-4f56-a425-ae90c070b53a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.062361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53670-3229-4f56-a425-ae90c070b53a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.066409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 442/891] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:12.067878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.068156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.068437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.087067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:12.088383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:12.116320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eab62-bcd3-40d0-ada9-15354291e5c3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.163256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.163897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.164186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.164477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.165190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.165493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.167487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.174534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 443/892] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:12.175362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.189573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 450/893] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:12.193663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.216795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.239171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.313602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd801-9833-4b67-9529-ed9bcf9928ac None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.346421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e186b5d-d60e-4182-a763-bde98f06dd8d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.356672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.357728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.358291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.358807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.359750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.360244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.361865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.368823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 444/894] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:12.369732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.426310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.427194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.427484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.427756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.429487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.431239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:12.448527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:12.450604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:12.452300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:12.455211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 451/895] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:12.456676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.528092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.536948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945e71e-0eb1-40a9-8461-016b63e4b0e0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.590871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.592510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.593023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.593804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.605530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.607094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.623749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:12.629458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:12.630461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:12.631348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:12.634316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 445/896] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:12.636930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.644982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832c342-ce62-4071-97b3-7091aec09ddd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.696210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.699424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.699711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.700000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.700729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.701037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.702655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.709546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 452/897] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:12.715861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.783184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daec06f-1f10-40cf-8b86-70f04ed26208 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.830797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.831356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.832231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.832511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.833582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.833895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.836194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.845313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cdae9-1464-4c58-8536-78ae93ace6a1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:12.853894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 446/898] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:12.861756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.881777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.896340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:12.902843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:12.913211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 453/899] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:12.914703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:12.938667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:12.999757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12352-3ba0-4a55-9547-1b713b235587 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:13.042574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:13.043679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:13.044184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:13.044662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:13.045708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] RBAC: Adding query filter params (role.id=7318de46958d496994150a3cbcacb1d0, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=86d945bffa964372bec61684a4a1e95a) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:13.045852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6e527-7f03-427b-9509-260b4dafb7ed None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:13.046417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:13.048292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:13.062632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 447/900] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/role_assignments?user.id=86d945bffa964372bec61684a4a1e95a&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=7318de46958d496994150a3cbcacb1d0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:13.089879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:13.090507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:13.090787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:13.091059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:13.091873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] RBAC: Adding query filter params (interface=public, service_id=28745eb520bc458780fe6cfaedf503bd, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:13.092184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:13.093607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f5ff-f1cc-4f2b-b2b8-c547591c7e0f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:13.105179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 454/901] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:16:12 2022] GET /identity/v3/endpoints?service_id=28745eb520bc458780fe6cfaedf503bd&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:13.106757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:13.204147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:13.232079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffb88d-f2bd-4561-a0fa-eb880afddefa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:13.274869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:13.275551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:13.275887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:13.276157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:13.277439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:13.278888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca65fc3-00b5-46ff-b38c-b5ed7e813164 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:13.290753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 448/902] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:13 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:16:14.322686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569f588-65e3-476b-a110-5cee4c083f8a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:14.323747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569f588-65e3-476b-a110-5cee4c083f8a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:14.324175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569f588-65e3-476b-a110-5cee4c083f8a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:14.327193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 455/903] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:14.336369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:14.336914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:14.337340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:14.358864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:14.360070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:14.437877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 449/904] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:14.462274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569f588-65e3-476b-a110-5cee4c083f8a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:14.635452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:14.658086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569f588-65e3-476b-a110-5cee4c083f8a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:14.723091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:14.724512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:14.724988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:14.725531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:14.726784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:14.729365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:14.754630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:14.755656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:14.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:14.757360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 456/905] 10.222.0.48 () {56 vars in 1132 bytes} [Fri May 27 09:16:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:14.764587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:14.911049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:14.954991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cc845-4f85-4a71-951d-a57fc64381e1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:14.996538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:14.997654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:14.998147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:14.998680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:14.999611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:15.000170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:15.001898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:15.008076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 450/906] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:15.013722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:15.214353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d00cd-d51f-4ed1-a35a-d77c43110814 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:15.279973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:15.281121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.281604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.282080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.283412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:15.283921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:15.286547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:15.296510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 457/907] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:15.307752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:15.516282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64da0-51cf-4550-8e8a-89ce6684e3e4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:15.605930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:15.607132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.607646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.608147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.609727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] RBAC: Adding query filter params (interface=public, service_id=28549afb6931460290862ed2199cc741, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:15.610342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:15.613118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cb2d8-bcfa-4ec7-8f6b-025e0f97f6cd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:15.622963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:15.628627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 451/908] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:16:15 2022] GET /identity/v3/endpoints?service_id=28549afb6931460290862ed2199cc741&interface=public®ion_id=RegionOne => generated 194 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:15.635228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:15.636630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cefa-197b-4db4-8f46-f91268be7ddc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.640688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cefa-197b-4db4-8f46-f91268be7ddc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.640688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56cefa-197b-4db4-8f46-f91268be7ddc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.640688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 452/909] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:15.644697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.645069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.645243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.658782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:15.670975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:15.670975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:15.806142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 453/910] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:15.829828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:15.865009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791aaf-869f-4a57-897f-2210f26cdbc6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:15.919917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:15.920593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.920945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.921224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.921928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:15.923360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dc6fe-8db0-49e6-a4b1-c330009d1a60 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:15.929740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 458/911] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:15 2022] GET /identity/v3/services => generated 1268 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:16:15.931120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ecf46-8ac0-4f13-8fcd-654cf43ee5b1 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.931412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ecf46-8ac0-4f13-8fcd-654cf43ee5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.931698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ecf46-8ac0-4f13-8fcd-654cf43ee5b1 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.933998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 459/912] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:15.944816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478be3-03a9-44fd-a8e7-7ca2effd7dc2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:15.945685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478be3-03a9-44fd-a8e7-7ca2effd7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:15.946134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478be3-03a9-44fd-a8e7-7ca2effd7dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:15.969358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0f478be3-03a9-44fd-a8e7-7ca2effd7dc2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:15.970804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0f478be3-03a9-44fd-a8e7-7ca2effd7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:16.009612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282333-4ab1-4ae0-bd91-3d4beabec2c4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.062739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.066262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 460/913] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:16.067507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094c8-a2c2-4c2b-850d-e9f1a90d74a3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.067790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094c8-a2c2-4c2b-850d-e9f1a90d74a3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.068065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2094c8-a2c2-4c2b-850d-e9f1a90d74a3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.069654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 461/914] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:16.082407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3da34-6459-492c-8976-733449a55abf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.082407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3da34-6459-492c-8976-733449a55abf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.082407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3da34-6459-492c-8976-733449a55abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.087693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.090132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.090458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.091824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.091824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.106864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-70d3da34-6459-492c-8976-733449a55abf None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:16.108609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-70d3da34-6459-492c-8976-733449a55abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:16.111380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:16.128055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:16.129320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:16.130167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:16.134654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 454/915] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:16.136790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.216077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 462/916] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:16.217699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3da34-6459-492c-8976-733449a55abf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.279093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c11aaf-faf7-450d-a931-a452f0d86253 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.318386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.319089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.319361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.319762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.320561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.321982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:16.334918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:16.335975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:16.337220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 455/917] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:16.338265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.379347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3da34-6459-492c-8976-733449a55abf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.429189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.438796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.439949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.440459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.440959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.442522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:16.443078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.446011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:16.449276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.468443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 463/918] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:16.468443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.489285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ee60-bb4f-490d-abd9-db7c8785805c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.550577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.551609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.551867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.552176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.552936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.554476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:16.615646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359fe33-cbac-46e7-a708-a7d8d6b39436 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.653188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.654188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.654188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.654509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.655184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:16.655489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.656872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:16.662598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 464/919] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:16.663436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.702235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 456/920] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:16 2022] POST /identity/v3/services => generated 241 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:16.702235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.769494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.828557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bba33b-7c94-453c-a723-bcc98258d27c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.882861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.883824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.884233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.884977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.892112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425735d-fcdd-4677-ba29-083fad8e3ebf None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:16.899834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.902131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:16.922012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:16.927847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:16.928924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:16.929820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:16.929820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:16.929820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:16.929820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:16.938309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 465/921] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:16.939557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:16.967033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:16.968420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:16.969152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:16.969730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:16.972254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:16.973243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:16.977186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:17.002161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 457/922] 10.222.0.48 () {56 vars in 1114 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:17.003949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:17.095737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940d28f-e178-4025-94b0-209ecec077bd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:17.107343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:17.138124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:17.139136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:17.139594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:17.140104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:17.141037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:17.141545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:17.143305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:17.164967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 466/923] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:17.166345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:17.243963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f102-32ed-43ba-a479-4fe25afdf4ce None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:17.310138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:17.312257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:17.312779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:17.313223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:17.315224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:17.319353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:17.325963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1b6a-74d8-44b0-b68e-f1938795a160 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:17.402050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:17.404620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:17.405107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:17.405650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:17.426324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:17.426324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:17.427374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 458/924] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:17.458795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:17.460685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:17.462668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 467/925] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:16:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:17.471472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:17.523987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:17.677292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74d6f-2593-411d-bfd1-f036f181433f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:17.751569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:17.752751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:17.753247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:17.753735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:17.755419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:17.755977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:17.758954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:17.796337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 459/926] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:17.808391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:17.871958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:18.021705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c3745-33c0-47c9-8b35-907d0963ea72 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:18.073653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:18.074821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:18.075263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:18.075669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:18.087256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:18.089008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:18.108721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:18.109696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:18.110668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:18.110668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:18.110668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:18.110668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:18.112750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 468/927] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:18.123608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:18.250609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:18.273857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8baaae-7cc2-4a76-b984-b5a01b82d443 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:18.324493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:18.325597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:18.326076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:18.326623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:18.327557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:18.328064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:18.329696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:18.343518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 460/928] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:18.354136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:18.547412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:18.573659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad217094-2f5f-4bff-85d5-5ea9df898821 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:18.624050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:18.624863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:18.625156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:18.625406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:18.635318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:18.636935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:18.657957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:18.659770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:18.661519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 469/929] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:18.674186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:18.914520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a3d4-afec-46ae-991f-b59cbe91ef9e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:18.966870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:18.967546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:18.967828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:18.970292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:18.970292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:18.970292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:18.976123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d213-30d9-4568-a407-1bb89827363c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:19.004964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 461/930] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:19.016540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:19.203272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae09fc-1f15-4ca5-9523-fb48a66eead2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:19.270313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:19.271888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.272376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.272856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/86d945bffa964372bec61684a4a1e95a/roles/7318de46958d496994150a3cbcacb1d0` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.277229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=86d945bffa964372bec61684a4a1e95a, role_id=7318de46958d496994150a3cbcacb1d0)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:19.280054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e0002-5101-4562-b1b8-e9a69319149b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:19.366755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc77a-87f4-491d-806d-1088141e8f00 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.367074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc77a-87f4-491d-806d-1088141e8f00 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.367308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecc77a-87f4-491d-806d-1088141e8f00 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.369102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 462/931] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:19.375906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.376310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.376472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.380464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:19.391635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 470/932] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:16:19 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/86d945bffa964372bec61684a4a1e95a/roles/7318de46958d496994150a3cbcacb1d0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:16:19.396045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:19.397303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:19.399872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:19.438281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f583b7-c14d-4614-9ed8-e3e3157cf9a5 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.438573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f583b7-c14d-4614-9ed8-e3e3157cf9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.438913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f583b7-c14d-4614-9ed8-e3e3157cf9a5 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.440934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 471/933] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:19.447937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.448924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.449109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.477467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:19.477467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:19.552998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 463/934] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:19.576447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:19.582646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 472/935] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:19.607937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:19.735469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddb86f-f196-4a1d-bf5d-3622113e2880 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:19.793399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:19.794566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.794843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.795117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.795799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:19.796047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedcebe-5e39-4a6d-b765-4440466194d1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:19.797295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:19.814887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:19.816843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:19.818368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 464/936] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:19.824067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:19.827007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:19.832116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:19.838773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:19.840112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.843436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.843436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.843436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:19.843436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:19.857117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:19.858268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 09:16:19.859162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:19.861770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 473/937] 10.222.0.48 () {56 vars in 1132 bytes} [Fri May 27 09:16:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:19.868075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:19.941806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c039c1-6697-40da-a955-a7b9394bb77b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:19.983146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:19.983714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:19.983998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:19.984271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:19.985016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:19.985322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:19.986905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:19.990586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd0d97-adbe-4c18-9b47-22f7a80c225a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:19.998046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 465/938] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:19.999094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:20.033914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.035272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.035700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.036115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.037046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:20.037559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.039173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.045583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 474/939] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:20.052147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:20.077901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:20.117907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288df14-a7ad-4c80-b1f8-9d5074c66551 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:20.128637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:20.157661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.158365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.158713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.158909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.159614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:20.159922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.161357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.172045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 466/940] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:20.172999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:20.182050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4cba-f9ff-4168-9651-c5b0b4755452 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:20.223444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.224453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.224938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.225422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.226534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:20.227026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.228633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.240758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 475/941] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:20.241980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:20.289789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53560b-bcb6-4699-9dea-d1df9c89e183 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:20.325840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:20.332826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.333507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.333791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.334064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.335707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] RBAC: Adding query filter params (interface=public, service_id=e03f6518c2f44fb4ab3f02ed4e623456, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:20.335707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.338056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.348069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 467/942] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:16:20 2022] GET /identity/v3/endpoints?service_id=e03f6518c2f44fb4ab3f02ed4e623456&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:20.349212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:20.367104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912e71-2a3f-4c23-a2eb-606e632098d0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:20.394662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:20.409377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.410386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.410821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.411263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.412208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.413792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3a89f-6620-4d18-b789-3ca679da2521 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.470622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0af444-080f-4761-a843-1ef9164452f8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:20.510319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:20.511533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:20.511810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:20.512082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:20.512769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:20.514147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002720b8-f283-4544-9f06-110e4f7a3428 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:20.521329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 468/943] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:20 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:16:20.597128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 476/944] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:21.214660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b17-58c9-489a-b213-040975d0647c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:21.215462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b17-58c9-489a-b213-040975d0647c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:21.215813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b17-58c9-489a-b213-040975d0647c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:21.218688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 469/945] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:21.227672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:21.228408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:21.228702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:21.259887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:21.262690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:21.404997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 477/946] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:21.447856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4760b17-58c9-489a-b213-040975d0647c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:21.605119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:21.660224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4760b17-58c9-489a-b213-040975d0647c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:21.732075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:21.733721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:21.734563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:21.735343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:21.736964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:21.740308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:21.743279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING keystone.server.flask.common [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 09:16:21.816592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.sql.core [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 09:16:21.816592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 09:16:21.816592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [parameters: {'id': 'c146dd995d774ce88b1db746aece6eb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 09:16:21.816592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80419) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c146dd995d774ce88b1db746aece6eb2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c146dd995d774ce88b1db746aece6eb2. May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application conn.query(q) May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:16:21.852355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result.read() May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 09:16:21.853976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application next(self.gen) May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application yield resource May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application next(self.gen) May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.session.rollback() May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.force_reraise() May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise self.value May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application session.commit() May 27 09:16:21.856998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._prepare_impl() May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.session.flush() May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._flush(objects) May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application compat.raise_( May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application flush_context.execute() May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 09:16:21.860949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rec.execute(self) May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 09:16:21.864719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application conn.query(q) May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result.read() May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:16:21.870458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [parameters: {'id': 'c146dd995d774ce88b1db746aece6eb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:21.879886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:21.883321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c146dd995d774ce88b1db746aece6eb2. May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:21.887476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 470/947] 10.222.0.48 () {60 vars in 1140 bytes} [Fri May 27 09:16:21 2022] POST /identity/v3/projects => generated 262 bytes in 432 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:21.903162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:22.094320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea401f0-0e66-4e50-9b2d-64c593fb3a31 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:22.096854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:22.132609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:22.133325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.133606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.133865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.141584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:22.143094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a82c5550-7de1-4cbc-910f-70bf4b86e0d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:22.158554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:22.160399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:22.163123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 478/948] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:22.176052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:22.393919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d446b-6978-4c30-9d59-2e8001948174 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:22.413172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394836c3-16e9-4b09-a43c-8ad68aa0a369 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.413913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394836c3-16e9-4b09-a43c-8ad68aa0a369 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.413913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394836c3-16e9-4b09-a43c-8ad68aa0a369 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.422268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 479/949] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:22.422268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.422268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.422268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.425145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:22.440504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:22.441722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:22.445878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:22.446664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.448590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.448590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.448810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:22.449150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:22.450908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:22.473251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 471/950] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:22.527590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 480/951] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:22.554145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:22.679344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5cfbf-675e-4d8c-9364-efc2a0b3c109 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:22.722390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:22.723247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.723582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.723852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.737616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-becdd150-43ba-4126-8196-5ee4a277696b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:22.746452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:22.749621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:22.751597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 472/952] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:22.758233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:22.872517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:22.884422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6dfad-6d91-431c-a743-9b81d3cce206 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:22.916514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c85f4-edc8-40ae-b97d-ca84958d5db7 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.917374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c85f4-edc8-40ae-b97d-ca84958d5db7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.917705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c85f4-edc8-40ae-b97d-ca84958d5db7 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.920611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 473/953] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:22.925061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.925425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.925718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.944543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:22.946298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:22.950861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:22.951539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:22.951816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:22.952087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:22.952834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:22.953134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:22.954536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:22.964778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 481/954] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:22.970394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:22.988083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.047350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 474/955] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:23.069848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.090667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7d60b-d79e-4027-bd87-e1568cf4a97d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.130837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.131584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.131915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.132195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.146575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.148301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:23.170691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:23.171683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:23.172617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:23.172617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:23.172617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:23.172617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:23.177582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 482/956] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:23.180885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.195823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12861f-b063-4562-874c-bdbb252d0bc4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.202640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.239412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.240544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.240990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.241420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.242305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.244114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:23.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:23.258729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:23.259609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:23.263501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 475/957] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:23.269466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.303592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.311015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2222e-af80-490a-8b17-767692d6bf06 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.353949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.354842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.355368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.355689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.356547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:23.356932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.359519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.378524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 483/958] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:23.379803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.404066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924d911-8029-4516-baa1-e9a7026d13c7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.451989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.452998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.453488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.454052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.456110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:23.457895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.459971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.472747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 476/959] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:23.477954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.513191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.583068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8421a4-61eb-4660-a035-1222df5fca83 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.603040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.613857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eaf40-9c4a-42e8-817c-6a5d69df669f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.657752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.658794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.659222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.659647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.660539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.660721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:23.661190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.662344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.662963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.663168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.663660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.670133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 477/960] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:23.679459 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.679831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.683714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:23.717021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:23.718102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 09:16:23.719213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:23.722843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 484/961] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:23.731957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.850993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c370-5a36-4f40-99ad-d539d8fde5bd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.868413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d4ce2-46f7-43ee-a411-d24eb4ed26f9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:23.881604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:23.891436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.892368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.892803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.897327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.897327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:23.897327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.897327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:23.921927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 485/962] 10.222.0.48 () {56 vars in 1149 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:23.923229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:23.938162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:23.939335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:23.939834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:23.940307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:23.941728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:23.944361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:24.047130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb468-ae97-4dc9-a9c3-58ee0f936549 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:24.089334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 478/963] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:24.105480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:24.106397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.106682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.106962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.173248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:24.196391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:24.198635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba33a-fc99-4d4d-a550-68a8b26c2c4c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:24.325800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 486/964] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:24.331687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:24.350117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:24.509417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11565c-6b8c-4884-8668-16d570346b91 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.509680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11565c-6b8c-4884-8668-16d570346b91 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.510091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11565c-6b8c-4884-8668-16d570346b91 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.511874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 487/965] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:24.521223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.523751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.524505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.525967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321c58-c57a-4cca-849c-70b633211b05 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:24.573464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:24.575623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:24.588127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:24.589417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.589706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.589974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.603615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:24.605757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:24.629554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:24.631636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:24.633346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:24.633346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:24.633346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:24.633346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:24.635682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 479/966] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:24.636643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:24.699689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 488/967] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:24.731218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:24.731996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:24.759264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:24.786404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582caf7-eccf-4ba3-aade-8bd8151c2e8d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:24.840910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:24.842118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.842674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.843158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.844168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:24.844820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:24.846644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:24.865829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 480/968] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:24.867796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:24.875168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf7b73-ca71-4bb1-828d-e18f091e3b7c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:24.927027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:24.928105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:24.928396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:24.928674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:24.929849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:24.931395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:24.949863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:24.950957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:24.957118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 489/969] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:24.959567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:25.003899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f87c8-e52a-416b-996c-e94da95544c0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:25.038178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:25.055310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:25.056133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.056412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.056698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.066720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:25.069149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:25.092555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:25.094024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:25.095942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:25.098567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 481/970] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:25.100961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:25.102147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:25.145458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a297-fc1e-469d-b735-22258ebcd853 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:25.214817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:25.216416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.217293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.218007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.219607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:25.220361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:25.223319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:25.238428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 490/971] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:25.242002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74321844-dd70-40db-be7d-f363cb779ce0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:25.242898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:25.301438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:25.302914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.303583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.304264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.305796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:25.306559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:25.310131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:25.334638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 482/972] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:25.343478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:25.391945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45068932-8d59-4427-8867-2c2b2e96d28c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:25.437303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:25.437821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:25.438013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.438299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.438585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.439840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:25.439840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:25.441287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458f828-38ce-4ab4-ba70-0bb5d3e1257f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:25.454258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 491/973] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:25.513813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6127-48ac-4e19-96b5-1eecb982e3ac None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:25.554805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:25.555776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.555776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.556061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.556854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] RBAC: Adding query filter params (role.id=7318de46958d496994150a3cbcacb1d0, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=86d945bffa964372bec61684a4a1e95a) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:25.557213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:25.558754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217418-ade4-430f-b5ff-789469d462a9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:25.566752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 483/974] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:25 2022] GET /identity/v3/role_assignments?user.id=86d945bffa964372bec61684a4a1e95a&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=7318de46958d496994150a3cbcacb1d0 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:25.678807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b2e5f-2072-4494-9a25-d6ead94f2e6d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.679789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b2e5f-2072-4494-9a25-d6ead94f2e6d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.679789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b2e5f-2072-4494-9a25-d6ead94f2e6d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.683586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 492/975] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:25.693160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:25.693661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:25.694156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:25.724708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:25.727058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:25.847336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 484/976] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:25.892319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b2e5f-2072-4494-9a25-d6ead94f2e6d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:25.899136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:26.083881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b2e5f-2072-4494-9a25-d6ead94f2e6d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:26.145099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:26.146234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:26.146762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:26.147344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:26.148753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:26.152056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:26.154565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING keystone.server.flask.common [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 09:16:26.256515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.sql.core [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 09:16:26.256515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 09:16:26.256515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [parameters: {'user_id': '83539e66a67540f193a3789eddfb7968', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 09:16:26.256515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() May 27 09:16:26.273285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:16:26.275059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application yield resource May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.rollback() May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.force_reraise() May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise self.value May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application session.commit() May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 09:16:26.278898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._prepare_impl() May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.flush() May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._flush(objects) May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application compat.raise_( May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application flush_context.execute() May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rec.execute(self) May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 09:16:26.282288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 09:16:26.285747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 09:16:26.287524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [parameters: {'user_id': '83539e66a67540f193a3789eddfb7968', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:26.289227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 09:16:26.291533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:26.295131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 493/977] 10.222.0.48 () {60 vars in 1131 bytes} [Fri May 27 09:16:25 2022] POST /identity/v3/users => generated 160 bytes in 385 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:26.296049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:26.333295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:26.406800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0488d-ecf9-4ae6-a211-731e66829b23 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:26.449559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:26.450450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:26.450901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:26.451183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:26.460196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:26.461771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f73503a2-22df-4b3b-aeac-7af352025ca0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:26.478660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:26.481287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:26.483690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 485/978] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:16:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:26.496143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:26.631494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:26.698009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1aadc-2406-4110-b418-de080b0ebd38 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:26.758774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:26.759891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:26.760382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:26.761439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:26.762994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:26.763563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:26.766767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce9ae1-2332-4da4-a287-1db71c4eb718 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:26.846101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 494/979] 10.222.0.48 () {56 vars in 1101 bytes} [Fri May 27 09:16:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:27.288064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7375b1-2369-443b-a8c1-0a8dda5363ac None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:27.289566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7375b1-2369-443b-a8c1-0a8dda5363ac None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:27.290360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7375b1-2369-443b-a8c1-0a8dda5363ac None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:27.293641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 486/980] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:27.303294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:27.303803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:27.304296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:27.334040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:27.336811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:27.461627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 495/981] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:27.485144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7375b1-2369-443b-a8c1-0a8dda5363ac None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:27.674303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:27.683405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7375b1-2369-443b-a8c1-0a8dda5363ac None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:27.742606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:27.743993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:27.744485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:27.745554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:27.746856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:27.749525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:27.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:27.775271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:27.776653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 487/982] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:27.782148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:27.846429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:27.904914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4049a-b0c5-46bd-bf93-7e23f3d88c71 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:27.941911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:27.942599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:27.942900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:27.943153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:27.943852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:27.944150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:27.945783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c121-def5-4bb9-8036-d757d68d4426 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:27.961040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 496/983] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:27.962718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:28.078183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3271-fe63-4238-907b-ddbee7d747ae None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.078863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3271-fe63-4238-907b-ddbee7d747ae None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.079122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3271-fe63-4238-907b-ddbee7d747ae None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.081725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 497/984] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:28.090616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.091219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.091791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.119731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:28.121861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:28.135075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780483-3c73-4cfc-b23c-40fb0706602e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:28.205779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:28.207004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.207508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.208142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.209309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:28.209863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:28.212445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:28.227549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 488/985] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:28.230077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:28.231789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 498/986] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:28.257663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:28.281724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:28.321070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:28.406105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0543f-aa79-4890-91be-87020a0f9b92 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:28.421163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39f406-c849-40bf-88e5-f0d6b56ffaa0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:28.461339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:28.462975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.463490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.464665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.466180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] RBAC: Adding query filter params (interface=admin, service_id=e03f6518c2f44fb4ab3f02ed4e623456, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:28.466764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:28.469398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:28.470009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:28.470722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.471169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.471331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.471986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:28.473376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ed696-2169-40ee-a48d-60055c1a9997 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:28.478623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 489/987] 10.222.0.48 () {56 vars in 1249 bytes} [Fri May 27 09:16:28 2022] GET /identity/v3/endpoints?service_id=e03f6518c2f44fb4ab3f02ed4e623456&interface=admin®ion_id=RegionOne => generated 193 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:28.484454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:28.600148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 499/988] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:28 2022] POST /identity/v3/services => generated 242 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:16:28.601496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b2f4-157b-44de-b81b-f48d49fe8789 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.601788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b2f4-157b-44de-b81b-f48d49fe8789 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.602064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b2f4-157b-44de-b81b-f48d49fe8789 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.603630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 500/989] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:28.609785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.610183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.615797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.621159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:28.629895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:28.631204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:28.640669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abaeb-d9b5-4ae9-ae1d-caa948afda21 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:28.712903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:28.714501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.714941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.715375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.716268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:28.717959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:28.726087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 490/990] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:28 2022] GET /identity/v3/services => generated 1727 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:16:28.749656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 501/991] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:28.791072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:28.943635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3cc-7b0a-4698-bd64-8a3cb7cbcd1d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:28.996264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:28.997370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:28.997855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:28.998403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] PATH_INFO: `/v3/services/image` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:28.999684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:29.002288 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:29.025280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:29.027331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:29.028365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 491/992] 10.222.0.48 () {56 vars in 1105 bytes} [Fri May 27 09:16:28 2022] GET /identity/v3/services/image => generated 86 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:29.034136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:29.095081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:29.165374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cd11e-c714-44e2-b5eb-0cbc90590713 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:29.216479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:29.217562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:29.218078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:29.218570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:29.219941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] RBAC: Adding query filter params (name=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:29.220477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:29.222708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e01280-f34c-49a4-8e5d-7a115f25a0db None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:29.232466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 502/993] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:29.238433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:29.256850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:29.418688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1eb5-91d6-4398-924f-3480c5017832 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:29.487553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:29.488580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:29.489063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:29.489540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:29.490904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:29.491416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:29.494026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b312f6-f35b-46c5-983c-32462844d8f5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:29.508781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 492/994] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:31.019106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4a10-d227-43f9-adf3-03c5c3fa4d09 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.019667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4a10-d227-43f9-adf3-03c5c3fa4d09 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.020119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4a10-d227-43f9-adf3-03c5c3fa4d09 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.022960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 503/995] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:31.029025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f078-401b-4796-b34d-5a590ab9466e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.029792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f078-401b-4796-b34d-5a590ab9466e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.029792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f078-401b-4796-b34d-5a590ab9466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.050832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e9f7f078-401b-4796-b34d-5a590ab9466e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:31.052048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e9f7f078-401b-4796-b34d-5a590ab9466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:31.159316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 493/996] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:31.183920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d4a10-d227-43f9-adf3-03c5c3fa4d09 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.258884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0135fb-c96b-43ae-a278-7eaf3bc765e4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.259344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0135fb-c96b-43ae-a278-7eaf3bc765e4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.259797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0135fb-c96b-43ae-a278-7eaf3bc765e4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.262531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 494/997] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:31.270971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.271334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.271807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.301217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:31.303425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:31.307625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:31.325070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d4a10-d227-43f9-adf3-03c5c3fa4d09 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:31.328427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:31.388070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:31.389810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.390327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.390836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.392101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:31.394713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:31.419041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:31.430769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:31.432587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 504/998] 10.222.0.48 () {56 vars in 1126 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:31.434169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 495/999] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:31.436658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.453181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.579176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c659-61ab-4686-b0a0-912bb762eb67 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:31.637801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:31.639229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.639731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.640218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.641451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:31.643022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840ad5-8162-416a-bce6-d7bb8f8ee8cc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:31.644092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:31.664706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:31.666751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:31.667984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 496/1000] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:31.676577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.681932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:31.708249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:31.709452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.709963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.710502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.711844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:31.712394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:31.715718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:31.724837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 505/1001] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:31.735323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.793863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:31.843844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc208db-3a1e-469c-86f2-2ea1818ef656 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:31.904371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:31.905400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:31.905803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:31.906278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:31.907504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:31.908008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:31.910390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:31.918663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 497/1002] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:31.927927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:31.933910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241f7b3-0d58-4151-8c18-d4e2c0dbca2f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.004271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.005333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.005849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.006496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.008355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:32.008839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.011656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.029716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 506/1003] 10.222.0.48 () {56 vars in 1122 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:32.032880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:32.035074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:32.079280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472cc50-588b-4a76-a26b-bf3e063583e9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.134725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.135699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.136057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.136510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.137896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] RBAC: Adding query filter params (type=compute) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:32.138505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.141145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.147853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 498/1004] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:32.153916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:32.191258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:32.198857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90e8b-ae7e-4262-99a6-44aaefea17de None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.239969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.240735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.241101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.241440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.242296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.243667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a18add-a7da-4033-bc90-3e17e177ba27 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.281367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f9a06-735b-4fca-8245-ad64103034db None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.311183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:32.316648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 507/1005] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:32.317948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396320de-d839-4529-96b0-980d26b6fc59 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.318263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396320de-d839-4529-96b0-980d26b6fc59 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.318543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396320de-d839-4529-96b0-980d26b6fc59 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.319936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 508/1006] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:32.326772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.327285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.327727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.328934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.330063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.330493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.330930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.332380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] RBAC: Adding query filter params (interface=public, service_id=ea2f0fa6a40a48bd92c463491f0f6529, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:32.332902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.335439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.344433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 499/1007] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:16:32 2022] GET /identity/v3/endpoints?service_id=ea2f0fa6a40a48bd92c463491f0f6529&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:32.350687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:32.363498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:32.364941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:32.474138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 509/1008] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:32.500459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:32.509706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a6b6-0ded-4cf6-bfeb-2640d8b77cc6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.555690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.555690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.556126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.556126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.557990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.559151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769733d-a674-4042-8b52-c732bfd0e215 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.568687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 500/1009] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:32 2022] GET /identity/v3/services => generated 1727 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:16:32.635218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:32.656654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfac30-6e8b-4011-8932-d703e4b6cc43 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:32.716497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:32.717300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:32.717745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:32.718084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:32.719119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:32.721443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815e2f4-23ee-4a56-98a9-bf5a7ee3471b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:32.821917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 510/1010] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:32 2022] POST /identity/v3/services => generated 242 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:16:34.650654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f98f-7606-4461-861c-8847f492a5bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:34.651237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f98f-7606-4461-861c-8847f492a5bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:34.651237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f98f-7606-4461-861c-8847f492a5bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:34.652830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 501/1011] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:34.658495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1088a8f-d58e-4e4b-8a42-63dee7190940 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:34.659019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1088a8f-d58e-4e4b-8a42-63dee7190940 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:34.659125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1088a8f-d58e-4e4b-8a42-63dee7190940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:34.678051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a1088a8f-d58e-4e4b-8a42-63dee7190940 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:34.679328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a1088a8f-d58e-4e4b-8a42-63dee7190940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:34.763832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 511/1012] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:34 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:34.786884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f98f-7606-4461-861c-8847f492a5bf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:34.846008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:34.867541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae9e8-7274-4aa4-9a81-1ffe3c52d49a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:34.867859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae9e8-7274-4aa4-9a81-1ffe3c52d49a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:34.868112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae9e8-7274-4aa4-9a81-1ffe3c52d49a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:34.869638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 512/1013] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:34.874927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:34.875218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:34.875499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:34.892504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:34.893697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f98f-7606-4461-861c-8847f492a5bf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:34.894079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:34.932627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c014be-605e-4067-b335-31655bd2919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:34.933431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c014be-605e-4067-b335-31655bd2919a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:34.933714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c014be-605e-4067-b335-31655bd2919a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:34.933989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c014be-605e-4067-b335-31655bd2919a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:34.935218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c014be-605e-4067-b335-31655bd2919a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:34.936664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c014be-605e-4067-b335-31655bd2919a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:34.942934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 502/1014] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:34.944044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c014be-605e-4067-b335-31655bd2919a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:34.964556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:34.970058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 513/1015] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:34 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:34.993440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.066726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c014be-605e-4067-b335-31655bd2919a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.092979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:35.107937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.109197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.109938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.110656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.111582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.112288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967935cb-5f22-4a32-beb8-bb8f2981bac1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.113469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8623e9f-5a0c-4843-91db-d713c070b2fa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.147880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.148671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.148948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.149219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.149868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.151911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:35.164156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:35.169679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.170611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 514/1016] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:35.171929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.202485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 503/1017] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:16:34 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:16:35.223780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25276f2-e38a-40bb-bb56-b75edbc94b63 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.224093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25276f2-e38a-40bb-bb56-b75edbc94b63 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.224353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25276f2-e38a-40bb-bb56-b75edbc94b63 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.225945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 504/1018] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:35.232107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.232421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.232696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.244056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:35.250529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:35.251779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:35.292031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce2c8e-27ce-426f-8817-888522f05c07 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.331263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.331948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.332230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.332530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.333232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:35.333536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.334119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 505/1019] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:35 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:35.334999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.346156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 515/1020] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:35.347886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.357469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.466817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4d155-2e29-442c-a955-cbff596d6c57 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.471097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:35.476136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b75b8f-ea24-4502-acae-7b547ce1ce36 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.505701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:35.508649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.509703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.510268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.511368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.512455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:35.512931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.514764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.515186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.515567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.515869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.516133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.516789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.518165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.522433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 506/1021] 10.222.0.48 () {56 vars in 1112 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:35.528726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:35.531656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:35.532689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:35.533537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:35.538077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 516/1022] 10.222.0.48 () {56 vars in 1105 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:35.541477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.644937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1feb-79df-4f9e-a7e4-67a77281b1fb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.657503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbadbec-0435-4810-b538-0c46f6d345ae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.683189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.683849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.684128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.684397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.685147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.686547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.696392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.697058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.697359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.697636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.698941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] RBAC: Adding query filter params (name=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:35.699289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.700736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.706419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 517/1023] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:35.712054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.770583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:35.781901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 507/1024] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:16:35.822098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895c76e-e9eb-4054-bc84-73a731847834 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:35.870480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:35.871598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:35.872066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:35.872520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:35.873460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:35.873958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:35.875619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:35.886243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 518/1025] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:35.888997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:35.917206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:36.008293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16d5a-7fc3-4c20-8ae1-f9c9ad467e0c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:36.045080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:36.045669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:36.045949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:36.046247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:36.047046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] RBAC: Adding query filter params (interface=public, service_id=0884221254bb44289dfab342b249d454, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:36.047364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:36.048753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117edec2-e990-4e2e-a9ab-56ed0b25ba69 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:36.054647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 508/1026] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:16:35 2022] GET /identity/v3/endpoints?service_id=0884221254bb44289dfab342b249d454&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:36.061051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:36.171585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85893c2-07d8-44cd-b045-650850ec8cf8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:36.189235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:36.208512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:36.209179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:36.209463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:36.209747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:36.210647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:36.212056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ec5b4-b9f2-4d80-aed3-635a0dec96bd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:36.219471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 519/1027] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:16:36 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:16:37.550571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52ce4-d1cc-42cb-9ac6-0eda1b835b22 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:37.551206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52ce4-d1cc-42cb-9ac6-0eda1b835b22 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:37.551206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52ce4-d1cc-42cb-9ac6-0eda1b835b22 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:37.552485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 509/1028] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:37.558408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:37.558903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:37.558993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:37.581068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:37.582267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:37.686487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 520/1029] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:37.724365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a52ce4-d1cc-42cb-9ac6-0eda1b835b22 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:37.849995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:37.865050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a52ce4-d1cc-42cb-9ac6-0eda1b835b22 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:37.906277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:37.907529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:37.908827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:37.909172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:37.911078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:37.913055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a6520-d668-4e9d-a676-f79ff9e49157 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:37.926230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 510/1030] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:37.926485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:38.044405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d618ea-4f7d-4e58-9733-f1637a31f92c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:38.068120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:38.113819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:38.114886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.115368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.115832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.116804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:38.118738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f32d5c-8848-4987-a8ff-d6ecec2d8b6e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:38.242233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 521/1031] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:16:37 2022] POST /identity/v3/users => generated 285 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:38.268153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a98bcf-bd8f-4374-9d83-962f755932e0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.268457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a98bcf-bd8f-4374-9d83-962f755932e0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.268698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a98bcf-bd8f-4374-9d83-962f755932e0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.270223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 511/1032] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:38.276610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ccb6-4b08-48c1-8011-d66c844f1131 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.276992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ccb6-4b08-48c1-8011-d66c844f1131 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.277204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ccb6-4b08-48c1-8011-d66c844f1131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.296856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d6a3ccb6-4b08-48c1-8011-d66c844f1131 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:38.298093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d6a3ccb6-4b08-48c1-8011-d66c844f1131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:38.405874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 522/1033] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:38.435400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a98bcf-bd8f-4374-9d83-962f755932e0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:38.523878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fece-fe2b-441b-ae7f-d182619fa675 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.524325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fece-fe2b-441b-ae7f-d182619fa675 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.524747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1fece-fe2b-441b-ae7f-d182619fa675 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.527744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 523/1034] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:38.533951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.534540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.534968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.559169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:38.560787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:38.585824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a98bcf-bd8f-4374-9d83-962f755932e0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:38.647053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 524/1035] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:38.648281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:38.649389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.649871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.650414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.672104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:38.681774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:38.687701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:38.689600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:38.696489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 512/1036] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:38.697571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:38.715862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:38.719978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:38.831489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e98f3-bcc9-4b69-ac8b-533b2a065662 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:38.836873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faed29a-7617-4750-96cb-d85652520e2d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:38.880215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:38.881006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.881294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.881567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] PATH_INFO: `/v3/services/image` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.882260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:38.883662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:38.890785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:38.892289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:38.892957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:38.893678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:38.895231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:38.895965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:38.897820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:38.899638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:38.901423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 525/1037] 10.222.0.48 () {56 vars in 1105 bytes} [Fri May 27 09:16:38 2022] GET /identity/v3/services/image => generated 86 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:38.903124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:38.905503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:38.922625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 513/1038] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:38.924436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.017250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.027829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c085c64-6e6e-45c5-9f0a-7036b694bf2f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.060718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.068625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.069292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.069626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.069863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.070852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] RBAC: Adding query filter params (name=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:39.071975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.072529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.077907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 526/1039] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:39.083746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.105191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f0dd1-ddcc-4da5-9896-f2d942064f9a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.143343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.144140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.144420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.144697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.156836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.158630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:39.177462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:39.178454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:39.185512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:39.185512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:39.185512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:39.185512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:39.185512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 514/1040] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:39.185986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.257610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c2fc5-0de2-4424-999d-98ae66eb5e2f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.316552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe19bf-a7ca-4da1-98aa-c245b7601fca None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.316799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.317714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.318154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.318625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.319802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:39.320604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.323602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.332616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 527/1041] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:39.339526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.355367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.361950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.362962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.363451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.363899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.364805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:39.365286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.366903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.375034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.386924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 515/1042] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:39.388206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.504114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce4c5-63aa-479f-92ad-47716918c014 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.509797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfad9e-fc2b-4220-a094-f5956f9982b6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.555281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.556089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.556353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.556624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.560133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.561157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.561643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.562136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.563540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.567498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.567685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.567777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:39.585025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:39.587426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:39.589408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 516/1043] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:16:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:39.592531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.645063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.680495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 528/1044] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:16:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:16:39.716003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c1d7-c391-49d0-84b1-c2db35ac96d2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.770541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.771326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.771609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.771882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:39.772817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:39.773126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:39.774703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:39.805041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 517/1045] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:16:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:39.806436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:39.900704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:39.935248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7844c0-2e92-4b0e-864f-b2879d7e53ba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:39.973388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:39.974370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:39.974813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:39.975244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.052284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:40.054055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:40.141167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:40.179277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 529/1046] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 09:16:40.189142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:40.344826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a392de8-1bc3-4f5b-97dc-8e5bcceedadc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:40.347967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:40.407104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:40.408498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:40.409013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:40.409524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.428137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:40.434323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-30991860-eca6-44e7-899f-9be1e147b5fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:40.465538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:40.475024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:40.476829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:40.476829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:40.476829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:40.476829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:40.476829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 518/1047] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:40.479815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:40.636158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb1c9d-b896-4983-828d-98998080c4b4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:40.670995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401338e6-b4bf-4596-acb0-662f0aa92d4d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:40.671524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401338e6-b4bf-4596-acb0-662f0aa92d4d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:40.671524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401338e6-b4bf-4596-acb0-662f0aa92d4d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.673134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 519/1048] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:40.677373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:40.678047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:40.678376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:40.678489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c779-278c-4813-9b97-37520611ca10 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:40.678661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.678762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c779-278c-4813-9b97-37520611ca10 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:40.678971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367c779-278c-4813-9b97-37520611ca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.679389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:40.679693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:40.681077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:40.698454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6367c779-278c-4813-9b97-37520611ca10 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:40.699805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6367c779-278c-4813-9b97-37520611ca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:40.703493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 530/1049] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:40.709543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:40.773852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:40.797513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 520/1050] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:40.821708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c779-278c-4813-9b97-37520611ca10 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:40.877273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eca9b6-27e7-486b-ba5c-035bc6894c9d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:40.920617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:40.921357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:40.921643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:40.921921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:40.930093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:40.931679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:40.949441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:40.950895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:40.952350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 531/1051] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:40.956851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:40.960933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:40.987768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367c779-278c-4813-9b97-37520611ca10 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.051302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.052817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.053302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.053958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.071820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:41.077349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:41.080070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eeb23-ee73-4c96-853d-3c6d306355e0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:41.092429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:41.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:41.094838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 521/1052] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:41.104503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:41.125628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.126775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.127102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.127367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.128196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:41.128718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:41.131484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:41.153547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 532/1053] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:41.160375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:41.237077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0cb55-f436-4bb4-b2ca-266ac3eb33fd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.285982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.286704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.286988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.287267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.288025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:41.288334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:41.289732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:41.301630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 522/1054] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:41.308024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:41.321104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:41.337568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912b5a2-e54f-448a-b1ec-36cbe06dffdc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.397231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.398371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.398872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.399360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.401007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] RBAC: Adding query filter params (role.id=7318de46958d496994150a3cbcacb1d0, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:41.401558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:41.404274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:41.414307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 533/1055] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:41 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=7318de46958d496994150a3cbcacb1d0 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:41.440070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc211437-c06a-40d6-9cdd-dc9932e29e77 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.490389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.491204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.491506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.491778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] PATH_INFO: `/v3/users/e852d78fb4c1487c805e88dc501ed0a7` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.513049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] RBAC: Authorizing `identity:get_user(user_id=e852d78fb4c1487c805e88dc501ed0a7)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:41.515062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:41.517478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 523/1056] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:41 2022] GET /identity/v3/users/e852d78fb4c1487c805e88dc501ed0a7 => generated 285 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:41.524901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:41.562499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:41.706178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448af5-c852-4470-a065-7b9a82dc7810 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:41.753760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:41.754622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:41.754906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:41.755222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:41.853385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:41.855547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053b4cc-53a4-4de7-9140-94bd130b5f70 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:41.889600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:42.047742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 534/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 09:16:42.049629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:42.073350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f505c-22df-4613-95dc-a72f5b12f073 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.073654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f505c-22df-4613-95dc-a72f5b12f073 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.073883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f505c-22df-4613-95dc-a72f5b12f073 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.075558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 535/1058] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:42.081503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.082073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.082073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.106412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:42.108267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:42.163214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:42.210885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604af7-e326-47a6-9d4f-4271b7e366d6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:42.217493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 536/1059] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:42.252034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:42.269815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:42.271234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.271701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.272171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.274076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:42.277132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:42.285690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 524/1060] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:16:42 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:42.287820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:42.331944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:42.410985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714619c-0b8f-40e9-b996-7455a659bdd0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:42.447646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a73b71-0a54-4d5c-8e0f-5ee4cde3bbec None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:42.451296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:42.452231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.452672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.452988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.503229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:42.503863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.503863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.504429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.505189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=31e8b694e70a43e6b740c3c37183e763, user.id=e852d78fb4c1487c805e88dc501ed0a7) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:42.505499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:42.507320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:42.509190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:42.527049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 525/1061] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:42 2022] GET /identity/v3/role_assignments?user.id=e852d78fb4c1487c805e88dc501ed0a7&scope.project.id=31e8b694e70a43e6b740c3c37183e763&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:42.571853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:42.573623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:42.648752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:42.742629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 537/1062] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:42.745480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:42.899188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ceb46-dfba-46f5-a6c3-665a6e0e4d8e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:42.962031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:42.963401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:42.964375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:42.964859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:42.984389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:42.987166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3c0168b0-902a-4c84-a63a-7456409814d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:43.016036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:43.017122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:43.018059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:43.018059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:43.018059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:43.018059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:43.024452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 526/1063] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:43.026781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:43.149539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db165d82-a786-4865-8db3-326b45092f93 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:43.198557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:43.199730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:43.200301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:43.200876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:43.202300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:43.202862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:43.205310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e537a-90af-4801-9d58-7696c25bb8ec None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:43.234261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 538/1064] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:43.689211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869db9-7cff-426e-9209-b4add62a395a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:43.689809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869db9-7cff-426e-9209-b4add62a395a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:43.689809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869db9-7cff-426e-9209-b4add62a395a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:43.691143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 527/1065] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:43.697191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:43.697569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:43.698407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:43.716697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:43.717884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:43.720125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:43.805439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 539/1066] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:43.829133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869db9-7cff-426e-9209-b4add62a395a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:43.899499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:43.960404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869db9-7cff-426e-9209-b4add62a395a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:43.996904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:43.997712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:43.997986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:43.998276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.009743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:44.018167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:44.020205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:44.021675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 528/1067] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:44.025719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.132851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea2b8c-6706-4078-bb65-f04b403e6b11 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:44.168730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:44.169396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.169674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.169941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.170722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:44.171024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:44.172410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:44.178374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:44.182010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 540/1068] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:44.188094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.299685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574149-34cf-4af8-b72e-593a70c44e40 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:44.311228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:44.336170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:44.336998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.337238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.337506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.349649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:44.351815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:44.370396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:44.371664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:44.372942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:44.372942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:44.372942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:44.372942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:44.372942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 529/1069] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:44.378404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.498861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e4715-ab4f-4ce9-8bf8-753324e9d6c8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:44.540206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:44.540884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.541160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.541428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.542339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:44.542645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:44.544062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:44.558652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 541/1070] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:44.564639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.684542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ee344-5671-4458-897d-e1006a189401 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:44.728697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:44.729693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.729886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.730075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.738525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:44.739998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-680e557f-c6c3-4e11-8781-17e2eac9a4d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:44.755580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:44.756758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:44.758299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 530/1071] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:44.763339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.827024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:44.871029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528d5f4-7a49-45d7-84b7-9437ce9c06fe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:44.876478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c7f-43eb-4dad-ab93-aa0d4e95b5d9 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.876754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c7f-43eb-4dad-ab93-aa0d4e95b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.877034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55c7f-43eb-4dad-ab93-aa0d4e95b5d9 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.878453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 531/1072] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:16:44.884121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.884405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.884688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.904195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:44.905421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:44.909200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:44.910044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:44.910404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:44.910721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:44.911609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:44.911996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:44.913507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:44.924477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:44.933935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 542/1073] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:44.940783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:44.991445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 532/1074] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:45.014577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.068673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93e34a-76a7-4271-9ac8-2fda5f30d30f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.088369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.108807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.109977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.110417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.110804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.119385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46060674-f6e9-4422-a85c-479c3d31d7ca None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.124902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.126614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:45.146898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:45.148383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:45.149696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:45.149696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:45.149696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:45.149696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:45.149696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 543/1075] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:45.154290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.160320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.161352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.161785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.162178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.175037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:45.183423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:45.185466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:45.186738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:45.189801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 533/1076] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:45.190788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.210845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.268974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f905a46-7fef-436e-b51c-e56679aa8164 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.308704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.309473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.309855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.310126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.310884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:45.311183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.312544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.312756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d4164-3643-4030-8326-f3cac68f1c27 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.324583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 544/1077] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:45.332622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.359111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.360152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.360634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.361092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.362411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:45.362928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.365210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.374841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 534/1078] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:45.381187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.381478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.456852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f5868-8d60-4654-b37d-60339062c589 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.460607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.486164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeaec2-b603-4053-9a0c-b393d3d465cd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.492349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.493135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.493412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.493680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.501253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.502841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:45.518250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:45.524735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:45.525706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 545/1079] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:45.526791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.526791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.526791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.526791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] PATH_INFO: `/v3/users/e852d78fb4c1487c805e88dc501ed0a7` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.527229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.527300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] RBAC: Authorizing `identity:get_user(user_id=e852d78fb4c1487c805e88dc501ed0a7)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.528376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.531004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 535/1080] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/users/e852d78fb4c1487c805e88dc501ed0a7 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:45.536343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.634300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.644537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830670a-1949-450e-a4b5-0101925b73ba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.654151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e608ef-3ebd-4e0c-bcb4-a5d024714745 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.681001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.681659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.681933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.682257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.683044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:45.683343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.684819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.688431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.689286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.689563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.689834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.691040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.692474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.694162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 536/1081] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:45.702300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 546/1082] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:45.702511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.703488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:45.714784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:45.806772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e223ce4-9444-47be-a0b0-a346dee1cb80 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.810398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30168d-9945-4ff0-b3a9-0f81a02bcc27 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:45.843074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.843994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.844217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.844488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763/users/e852d78fb4c1487c805e88dc501ed0a7/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.846465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] RBAC: Authorizing `identity:create_grant(project_id=31e8b694e70a43e6b740c3c37183e763, user_id=e852d78fb4c1487c805e88dc501ed0a7, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.846599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:45.847344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:45.847620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:45.847886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e8a29-d304-416a-b39c-81231523c33a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.847993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/7318de46958d496994150a3cbcacb1d0` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:45.850087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=55612ce26632413ebd0dfd5fc081b656, role_id=7318de46958d496994150a3cbcacb1d0)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:45.851537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364b045-60df-4927-9d3d-8ebde8bd6ddf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:45.941763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 537/1083] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:16:45 2022] PUT /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763/users/e852d78fb4c1487c805e88dc501ed0a7/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:16:45.971558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 547/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:16:45 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/55612ce26632413ebd0dfd5fc081b656/roles/7318de46958d496994150a3cbcacb1d0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:16:48.223932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6b75-6f93-4894-9435-d9a779df9880 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.224678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6b75-6f93-4894-9435-d9a779df9880 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.224678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf6b75-6f93-4894-9435-d9a779df9880 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.226080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 538/1085] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:48.230649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd47d1-5d41-4bd7-9025-c30718ccdf75 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.231147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd47d1-5d41-4bd7-9025-c30718ccdf75 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.231378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.231485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd47d1-5d41-4bd7-9025-c30718ccdf75 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.231665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.231938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.233764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 548/1086] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:48.239985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.240269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.240539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.250777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:48.252122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:48.258335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:48.259643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:48.263811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.336417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.344284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 539/1087] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:48.345130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 549/1088] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:16:48.367747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.368784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.478469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ce38c-48aa-490b-aa6d-2b2fcdf87d17 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.526840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.527874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.528243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.528608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.539330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a002-5886-4372-9239-6e5becef48c8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.541619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:48.549741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:48.550821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:48.551896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 540/1089] 10.222.0.48 () {56 vars in 1120 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:48.557816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.598632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.599438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.599714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.599984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.609089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.612916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:16:48.627311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:48.628373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:48.629222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 550/1090] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:16:48.635900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.637812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.684712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d38a-4f36-4c13-8129-9a8e47c0611d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.725870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.726677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.726950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.727229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.727972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:48.728273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:48.730303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:48.745902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 541/1091] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:48.752894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.755063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8320-68d1-46ce-a02e-75555f376d04 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.790214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.790879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.791160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.791963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.792713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:48.793016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:48.794414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:48.803066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 551/1092] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:48.810420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.861469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964457f-fee5-4eb3-87e2-45a23c9eea59 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.873202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.885251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:48.897375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.898272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.898576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.898948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.909923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:48.911433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:48.913758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a136b-8551-427c-bdfb-aa08bbd88b22 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:48.930215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:48.931180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:48.932732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:48.932732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:48.932732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:48.932732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:48.932732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 542/1093] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:48.938082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:48.953523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:48.954418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:48.954717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:48.956539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] PATH_INFO: `/v3/users/e852d78fb4c1487c805e88dc501ed0a7` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:48.956539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] RBAC: Authorizing `identity:get_user(user_id=e852d78fb4c1487c805e88dc501ed0a7)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:48.957624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:48.959333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 552/1094] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/users/e852d78fb4c1487c805e88dc501ed0a7 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:48.965181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.055599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1587366-e13c-4540-9882-3cc1bdbfdd98 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.082384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eac741-df9a-4b78-bb59-8cab983068c9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.092607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.093276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.093561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.093838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.094591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:49.094900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.096405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.114532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 543/1095] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:49.115634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.131649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.140410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.141536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.142044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.142466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.150081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.219059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330447e-c0fb-45f6-9541-63b9d037d190 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.232403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.233873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.255183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.255979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.256260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.256529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.264481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.265954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:16:49.282735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:49.284019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:16:49.284982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 544/1096] 10.222.0.48 () {56 vars in 1093 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:49.289897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.342384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 553/1097] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 09:16:49.348752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.381886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.393562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.400434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58f86d-2e91-4092-9641-a21c74c3608b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.439372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.440067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.440351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.440629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.441496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:49.441807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.443317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.461397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c4176-8d56-436a-a607-48f6d82d4491 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.464913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 545/1098] 10.222.0.48 () {56 vars in 1133 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:49.470939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.503995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.504900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.505180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.505454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.506832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.508372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.510710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 554/1099] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:49.517359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.588939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2ce0-1626-4de2-9164-4ddcf8ff9574 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.627881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.628267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.628267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.628267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.639132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.643250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5416c7-5d0f-4b3b-aefa-bddd73987928 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.654180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:49.679694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.680472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.680748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.681026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:49.681867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=31e8b694e70a43e6b740c3c37183e763, user.id=e852d78fb4c1487c805e88dc501ed0a7) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:49.682240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.683842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.690191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 555/1100] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/role_assignments?user.id=e852d78fb4c1487c805e88dc501ed0a7&scope.project.id=31e8b694e70a43e6b740c3c37183e763&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:16:49.709458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:49.711040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:49.833281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 546/1101] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 09:16:49.840900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:49.958630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88c58-a1fb-43d8-9895-420feba0f8cb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:49.994170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:49.995513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:49.995792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:49.996065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:50.008039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:50.010014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:16:50.029417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:50.030455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:50.031472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:16:50.031472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:16:50.031472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:16:50.031472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:50.031472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 556/1102] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:50.036736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:50.097550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:50.149668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd9ee-ee49-442b-a774-58cc0ca0c5b2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:50.189815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:50.190758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:50.191226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:50.191666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:50.192626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:50.193097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:50.195380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:50.215681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 547/1103] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:50.221238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:50.238657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:50.343376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bd19-f041-40e4-a3f1-da47fa639dfc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:50.378326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:50.379104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:50.379401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:50.379649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:50.386602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:50.387897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:16:50.401594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:16:50.402736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:16:50.403859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:16:50.406360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 557/1104] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:16:50.411066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:50.512677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adba70-b0da-4654-9a37-92bfb74a2311 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:50.526027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:50.550416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:50.551154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:50.551438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:50.551714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:50.552595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:50.552904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:50.554369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3f0e4-95cc-41ab-8d8b-c1592a573013 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:50.567793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 548/1105] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:50.575818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:50.652847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:50.677864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138e1da-050a-4a30-af80-1e6c6deb67c9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:50.711765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:50.712403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:50.712683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:50.712957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:50.714340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] RBAC: Adding query filter params (role.id=7318de46958d496994150a3cbcacb1d0, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=55612ce26632413ebd0dfd5fc081b656) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:50.714652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:50.716059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41e27b-b0eb-4764-b8b6-090dd24a8505 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:50.721773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 558/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:50 2022] GET /identity/v3/role_assignments?user.id=55612ce26632413ebd0dfd5fc081b656&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=7318de46958d496994150a3cbcacb1d0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:51.981620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7dea-5bad-40b5-b598-5897f38d50bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:51.982157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7dea-5bad-40b5-b598-5897f38d50bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:51.982157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7dea-5bad-40b5-b598-5897f38d50bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:51.983418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 549/1107] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:51.988629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:51.989072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:51.989178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:52.006381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:52.007559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:52.072043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 559/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:52.096879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7dea-5bad-40b5-b598-5897f38d50bf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:52.193898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7dea-5bad-40b5-b598-5897f38d50bf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:52.226458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:52.227227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:52.227502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:52.227771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:52.229478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:52.230928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022b935-86c8-4b70-817e-1c05910c1110 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:52.232555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 550/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:16:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:52.238157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:52.302772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:52.334105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23606-f4b0-4537-b440-71f31ae65234 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:52.367863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:52.368515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:52.368795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:52.369065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:52.369806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:52.371254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f551e-4af7-4944-b960-86a84361cb95 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:52.465799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 560/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:16:52 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:16:54.714567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a301-87f0-4c17-9005-1a744b4e57e6 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:54.715083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a301-87f0-4c17-9005-1a744b4e57e6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:54.715196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a301-87f0-4c17-9005-1a744b4e57e6 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:54.716457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 551/1111] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:54.721697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:54.722132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:54.722262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:54.739314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:54.740491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:54.810626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 561/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:54.834629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a301-87f0-4c17-9005-1a744b4e57e6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:54.876755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:54.933235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a301-87f0-4c17-9005-1a744b4e57e6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:54.966978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:54.967823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:54.968100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:54.968371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] PATH_INFO: `/v3/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:54.969821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] RBAC: Authorizing `identity:get_role(role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:54.971238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:54.972609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 552/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:54 2022] GET /identity/v3/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:54.978347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:55.031848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:55.080543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc2c8-b3a0-4fea-880a-5558b18272c1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:55.113293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:55.114079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:55.114385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:55.114660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] PATH_INFO: `/v3/users/50e46e571aa84e768b2c251b0557e0a0` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:55.127454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] RBAC: Authorizing `identity:get_user(user_id=50e46e571aa84e768b2c251b0557e0a0)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:55.128880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:55.130531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 562/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:54 2022] GET /identity/v3/users/50e46e571aa84e768b2c251b0557e0a0 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:55.136813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:55.233061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c1239-e45f-4185-bec4-7efcd48bd1c9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:55.241589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:55.266290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:55.266933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:55.267214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:55.267485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:55.334886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:55.336351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:55.431015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 553/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:16:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:16:55.437430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:55.534300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ac96a-4f6e-4c2d-b70b-4d81107d9090 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:55.547235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:55.567814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:55.568662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:55.568942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:55.569213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:55.570662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:55.572092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74bf3a-770f-4788-8d20-79ac6ed6f3c0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:55.573808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 563/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:16:55 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:55.579795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:55.589732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:55.675563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2367e-b282-4647-8891-5c31cb648472 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:55.708625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:55.709197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:55.709477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:55.709749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:55.710595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=31e8b694e70a43e6b740c3c37183e763, user.id=50e46e571aa84e768b2c251b0557e0a0) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:16:55.710905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:55.712356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bba9e9-7d6c-4b0f-9ce1-b720fe29448f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:55.717754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 554/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:16:55 2022] GET /identity/v3/role_assignments?user.id=50e46e571aa84e768b2c251b0557e0a0&scope.project.id=31e8b694e70a43e6b740c3c37183e763&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:57.904026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195c6b3-3ea0-4a86-9bff-6c3fed726aea None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:57.904542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195c6b3-3ea0-4a86-9bff-6c3fed726aea None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:57.904542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195c6b3-3ea0-4a86-9bff-6c3fed726aea None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:57.905829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 564/1118] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:16:57.911141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:57.911561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:57.911697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:57.928055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:16:57.929227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:16:57.993485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 555/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:16:58.016046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195c6b3-3ea0-4a86-9bff-6c3fed726aea None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:58.117113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195c6b3-3ea0-4a86-9bff-6c3fed726aea None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:58.150214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:58.151079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:58.151360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:58.151641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] PATH_INFO: `/v3/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:58.153101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] RBAC: Authorizing `identity:get_role(role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:58.154539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:58.155864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 565/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:58 2022] GET /identity/v3/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:58.161766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:58.164004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:58.259168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f37cb-6250-4d83-a451-47aa64bbab43 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:58.291936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:58.292784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:58.293061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:58.293331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] PATH_INFO: `/v3/users/50e46e571aa84e768b2c251b0557e0a0` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:58.294535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] RBAC: Authorizing `identity:get_user(user_id=50e46e571aa84e768b2c251b0557e0a0)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:58.295960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:58.297561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 556/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:16:58 2022] GET /identity/v3/users/50e46e571aa84e768b2c251b0557e0a0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:58.303339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:58.351841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:58.400672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ba367-bdc5-4d43-80f6-7745e68886ba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:58.434077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:58.434887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:58.435165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:58.435433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:58.436586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:58.438007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de57e8-3225-44a7-b685-261a568a37b8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:58.439812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 566/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:16:58 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:16:58.445997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:16:58.514160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:16:58.543614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2b77f-5b10-48ad-a5c8-4ae7a404756e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:16:58.577237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:16:58.578121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:16:58.578426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:16:58.578697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763/users/50e46e571aa84e768b2c251b0557e0a0/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:16:58.581091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=31e8b694e70a43e6b740c3c37183e763, user_id=50e46e571aa84e768b2c251b0557e0a0, role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:16:58.583003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a7ae3-a746-4edd-8d3d-00cbde6b63d8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:16:58.660674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 557/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:16:58 2022] PUT /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763/users/50e46e571aa84e768b2c251b0557e0a0/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:17:00.857035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c27-a7b5-48e0-bcf9-7f5ba25745c6 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:00.857534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c27-a7b5-48e0-bcf9-7f5ba25745c6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:00.857534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c27-a7b5-48e0-bcf9-7f5ba25745c6 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:00.858896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 567/1124] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:00.864123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:00.864501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:00.864667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:00.881604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:00.882812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:00.947514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 558/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:00.972156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d4c27-a7b5-48e0-bcf9-7f5ba25745c6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:01.069904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d4c27-a7b5-48e0-bcf9-7f5ba25745c6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:01.085957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:01.104172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] PATH_INFO: `/v3/roles/c008d0edf4ca47f8b86797d624e39a77` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] RBAC: Authorizing `identity:get_role(role_id=c008d0edf4ca47f8b86797d624e39a77)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:01.117672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 568/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:00 2022] GET /identity/v3/roles/c008d0edf4ca47f8b86797d624e39a77 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:01.118174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:01.174013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:01.215895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420bfba-96ba-42f8-a564-68a9a497a228 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:01.248943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:01.249794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:01.250070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:01.250370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] PATH_INFO: `/v3/users/50e46e571aa84e768b2c251b0557e0a0` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:01.251537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] RBAC: Authorizing `identity:get_user(user_id=50e46e571aa84e768b2c251b0557e0a0)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:01.252974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:01.254621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 559/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:01 2022] GET /identity/v3/users/50e46e571aa84e768b2c251b0557e0a0 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:01.261036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:01.356192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e1ed0-520c-4197-839e-0931aafd436a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:01.389016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:01.389573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:01.389851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:01.390117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:01.438830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:01.454100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:01.455552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:01.549493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 569/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:01.555309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:01.655333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cbab-08bd-4e00-a09b-6f964a1de4ff None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:01.680785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:01.688975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] PATH_INFO: `/v3/projects/31e8b694e70a43e6b740c3c37183e763` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] RBAC: Authorizing `identity:get_project(project_id=31e8b694e70a43e6b740c3c37183e763)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de554f8b-26c2-4a27-9385-35aba92f8a58 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:01.695588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 560/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:01 2022] GET /identity/v3/projects/31e8b694e70a43e6b740c3c37183e763 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:01.700932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:01.787599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:01.803267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685cd95-e8a6-46ac-9eaf-a26bd458f050 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:01.836942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:01.837584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:01.837859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:01.838685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:01.839507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] RBAC: Adding query filter params (role.id=c008d0edf4ca47f8b86797d624e39a77, scope.project.id=31e8b694e70a43e6b740c3c37183e763, user.id=50e46e571aa84e768b2c251b0557e0a0) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:01.839809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:01.841215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a832aac-0ce8-413f-a120-2086c750680c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:01.846974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 570/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:17:01 2022] GET /identity/v3/role_assignments?user.id=50e46e571aa84e768b2c251b0557e0a0&scope.project.id=31e8b694e70a43e6b740c3c37183e763&role.id=c008d0edf4ca47f8b86797d624e39a77 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:04.063270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d5a5-3129-4af8-8626-21a3b04c07e2 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:04.063823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d5a5-3129-4af8-8626-21a3b04c07e2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:04.063823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d5a5-3129-4af8-8626-21a3b04c07e2 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:04.065173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 561/1131] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:04.070629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:04.071064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:04.071161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:04.087355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:04.089004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:04.155491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 571/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:04.179017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828d5a5-3129-4af8-8626-21a3b04c07e2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:04.277995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828d5a5-3129-4af8-8626-21a3b04c07e2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:04.310631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:04.311404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:04.311682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:04.311952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:04.313137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:04.314645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aceea9b-380d-478c-bacc-bad8893d86c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:04.316197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 562/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:17:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:04.322328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:04.396417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:04.417593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26032bf-401c-4bf7-ba2d-df6122c1b758 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:04.451150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:04.451797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:04.452079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:04.452351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:04.453050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:04.454548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c75e2-29f4-46ee-8976-3a3351c9e88d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:04.565471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 572/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:17:04 2022] POST /identity/v3/projects => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:17:06.912679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4057-d68f-4b2d-b32d-a4fa44bca1db None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:06.913197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4057-d68f-4b2d-b32d-a4fa44bca1db None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:06.913197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe4057-d68f-4b2d-b32d-a4fa44bca1db None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:06.914536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 563/1135] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:06.919887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:06.920310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:06.920418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:06.945795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:06.946994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:07.012447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 573/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:17:07.037740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe4057-d68f-4b2d-b32d-a4fa44bca1db None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:07.099777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:07.138402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe4057-d68f-4b2d-b32d-a4fa44bca1db None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:07.175430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:07.176143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:07.176426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:07.176703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:07.177871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:07.179358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe559d-8b75-45da-9883-21e34521687a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:07.180987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 564/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:07.187416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:07.243868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:07.284231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42f177-1a03-475b-9486-d411de6eedd2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:07.316809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:07.317453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:07.317730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:07.317999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:07.318767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:07.320424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d486cac-acee-43bb-971f-9547797cb20b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:07.403814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 574/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:17:07 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:17:09.640136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2acaa9-c1cb-4629-bccb-e3ab09caca74 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:09.640702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2acaa9-c1cb-4629-bccb-e3ab09caca74 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:09.640702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2acaa9-c1cb-4629-bccb-e3ab09caca74 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:09.642018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 565/1139] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:09.647229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:09.647692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:09.647795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:09.664421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:09.665607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:09.732224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 575/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:09.755850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2acaa9-c1cb-4629-bccb-e3ab09caca74 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:09.853510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2acaa9-c1cb-4629-bccb-e3ab09caca74 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:09.881705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:09.886299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:09.887067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:09.887432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:09.887607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:09.897181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:09.904746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:09.905890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:09.907019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 566/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:09.912128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:09.957057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:10.007813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ca7c2-3daa-44c1-b747-a8c75c5c9986 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:10.040064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:10.040714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:10.040993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:10.041796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:10.042557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:10.042861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:10.044218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:10.052093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 576/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:10.058443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:10.156224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd2cd5-09c4-4e3b-ad08-0dd68714adf3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:10.188499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:10.189354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:10.189630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:10.189898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] PATH_INFO: `/v3/users/8528e0fe23a94b4a863e0ffc9d52544e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:10.202920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] RBAC: Authorizing `identity:get_user(user_id=8528e0fe23a94b4a863e0ffc9d52544e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:10.204361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:10.205986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 567/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:10 2022] GET /identity/v3/users/8528e0fe23a94b4a863e0ffc9d52544e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:10.211778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:10.308609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc421c33-1baa-4b28-bf4d-9a34b66051dc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:10.317063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:10.341233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:10.341863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:10.342139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:10.342434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:10.406845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:10.408201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:10.501979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 577/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:10.507832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:10.556944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:10.607043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be3441-d143-48d4-80a5-339d5d675083 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:10.639261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:10.640105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:10.640391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:10.640923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] PATH_INFO: `/v3/projects/76be29a4302d4837a1c931ae82c2a27f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:10.641804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] RBAC: Authorizing `identity:get_project(project_id=76be29a4302d4837a1c931ae82c2a27f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:10.643280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8cb79-0e5c-4a20-92e7-e679ec2925a5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:10.644957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 568/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:10 2022] GET /identity/v3/projects/76be29a4302d4837a1c931ae82c2a27f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:10.651542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:10.661628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:10.748459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aabf4-d28f-44b3-a82f-0d2834aabbe1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:10.781869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:10.782528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:10.782811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:10.783082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:10.783896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=76be29a4302d4837a1c931ae82c2a27f, user.id=8528e0fe23a94b4a863e0ffc9d52544e) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:10.784206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:10.785628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019b244-62ea-494b-bce6-36025aef6cd0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:10.791035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 578/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:17:10 2022] GET /identity/v3/role_assignments?user.id=8528e0fe23a94b4a863e0ffc9d52544e&scope.project.id=76be29a4302d4837a1c931ae82c2a27f&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:12.988409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bbf33-6442-40bc-a8eb-bcfb93b201e2 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:12.988922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bbf33-6442-40bc-a8eb-bcfb93b201e2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:12.988922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bbf33-6442-40bc-a8eb-bcfb93b201e2 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:12.990254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 569/1147] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:12.995973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:12.996360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:12.996529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.012910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:13.014084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:13.078759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 579/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:13.101426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bbf33-6442-40bc-a8eb-bcfb93b201e2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:13.200560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bbf33-6442-40bc-a8eb-bcfb93b201e2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:13.218837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:13.236044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:13.236879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:13.237159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:13.237431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.247405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:13.255392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:13.256558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:13.257498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 570/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:13.263333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:13.265516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:13.363266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec50a8-231a-4cf1-bdbc-a99ad6bfc8cb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:13.395205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:13.395854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:13.396130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:13.396398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.397243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:13.397548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:13.398926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:13.407370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 580/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:13.413329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:13.511388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67213a53-08d3-4bb2-8d48-dac7922ec48a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:13.544078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:13.544968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:13.545250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:13.545524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] PATH_INFO: `/v3/users/8528e0fe23a94b4a863e0ffc9d52544e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.546731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8528e0fe23a94b4a863e0ffc9d52544e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:13.548171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:13.549789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 571/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:13 2022] GET /identity/v3/users/8528e0fe23a94b4a863e0ffc9d52544e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:13.555413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:13.616063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:13.652654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610998a-ad84-48df-a1f2-be0ae1bd91be None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:13.684965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:13.685791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:13.686068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:13.686367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] PATH_INFO: `/v3/projects/76be29a4302d4837a1c931ae82c2a27f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.687516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] RBAC: Authorizing `identity:get_project(project_id=76be29a4302d4837a1c931ae82c2a27f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:13.688931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e061f4-7893-4d7a-931f-8c4ac8d609af None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:13.690626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 581/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:13 2022] GET /identity/v3/projects/76be29a4302d4837a1c931ae82c2a27f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:13.696815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:13.744326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:13.797764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e5b20-d885-430f-88dc-fe5fadb718d5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:13.830106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:13.831006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:13.831279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:13.831545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] PATH_INFO: `/v3/projects/76be29a4302d4837a1c931ae82c2a27f/users/8528e0fe23a94b4a863e0ffc9d52544e/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:13.833449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=76be29a4302d4837a1c931ae82c2a27f, user_id=8528e0fe23a94b4a863e0ffc9d52544e, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:13.834901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86240-d293-47b2-abdc-3c9fbd730fc7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:13.921401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 572/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:17:13 2022] PUT /identity/v3/projects/76be29a4302d4837a1c931ae82c2a27f/users/8528e0fe23a94b4a863e0ffc9d52544e/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:17:16.093843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a475d9-fe5a-4b0c-812c-e38df9ccc78e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.094404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a475d9-fe5a-4b0c-812c-e38df9ccc78e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.094404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a475d9-fe5a-4b0c-812c-e38df9ccc78e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.095751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 582/1154] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:16.101114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.101512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.101664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.117988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:16.119712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:16.185863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 573/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:16.210103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a475d9-fe5a-4b0c-812c-e38df9ccc78e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:16.309201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a475d9-fe5a-4b0c-812c-e38df9ccc78e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:16.337287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:16.341902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:16.342707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.342994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.343263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.356218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:16.361131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:16.362277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:17:16.363145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 583/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:16.368336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:16.402334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:16.464630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e027-209e-4eb8-9ca4-117bc9c8e4d6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:16.498581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:16.499242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.499521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.499793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.500526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:16.500831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:16.502217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:16.510132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 574/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:16.515967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:16.613176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8aec63-328e-4016-bfb0-c83d8353b25e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:16.645839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:16.646730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.647011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.647281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] PATH_INFO: `/v3/users/8528e0fe23a94b4a863e0ffc9d52544e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.648453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] RBAC: Authorizing `identity:get_user(user_id=8528e0fe23a94b4a863e0ffc9d52544e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:16.649879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:16.651536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 584/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:16 2022] GET /identity/v3/users/8528e0fe23a94b4a863e0ffc9d52544e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:16.657274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:16.750791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:16.754473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0840b1-658c-45d4-8a1c-64448aef81f1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:16.786909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:16.787553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:16.787836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:16.788111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:16.855397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:16.856905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:16.948428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:16.952781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 575/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:16.958771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:17.017068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:17.058057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05715fab-b3ff-4b56-9238-6f3a441af412 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:17.095438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:17.096302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:17.096584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:17.096858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] PATH_INFO: `/v3/projects/76be29a4302d4837a1c931ae82c2a27f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:17.098064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] RBAC: Authorizing `identity:get_project(project_id=76be29a4302d4837a1c931ae82c2a27f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:17.099534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942be8-532a-4731-a947-53ca4b32e636 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:17.101241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 585/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:16 2022] GET /identity/v3/projects/76be29a4302d4837a1c931ae82c2a27f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:17.107859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:17.203712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e7cf7-7ca2-41f1-b205-343a938d0476 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:17.237430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:17.238072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:17.238387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:17.238670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:17.239499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=76be29a4302d4837a1c931ae82c2a27f, user.id=8528e0fe23a94b4a863e0ffc9d52544e) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:17.239809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:17.241252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f5b4-559d-4f10-b190-31adca001df9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:17.248091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 576/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:17:17 2022] GET /identity/v3/role_assignments?user.id=8528e0fe23a94b4a863e0ffc9d52544e&scope.project.id=76be29a4302d4837a1c931ae82c2a27f&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:19.485565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4dc4-e163-45bf-8ad6-b39a7fb9e5d0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:19.486095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4dc4-e163-45bf-8ad6-b39a7fb9e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:19.486095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4dc4-e163-45bf-8ad6-b39a7fb9e5d0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:19.487475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 586/1162] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:19.492638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:19.493017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:19.493186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:19.509466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:19.511178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:19.565127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:19.575984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 577/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:19.598716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf4dc4-e163-45bf-8ad6-b39a7fb9e5d0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:19.695258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf4dc4-e163-45bf-8ad6-b39a7fb9e5d0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:19.721999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:19.729505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:19.735012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:19.735012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:19.735012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:19.795370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:19.796808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:19.891542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 587/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:19.897537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:19.992804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f100e72-750a-4cf5-9ee9-f38827402b8e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:20.026269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:20.027058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:20.027338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:20.027609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:20.034096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:20.035574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7eb7522d-74ce-44b8-bd8d-611a6b893edc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:20.049489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:20.050716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:20.051757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 578/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:17:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:20.057124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:20.084260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:20.152181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae9233-4d54-44bc-98a1-779971d0f724 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:20.184688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:20.185330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:20.185605 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:20.185873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:20.186607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:20.186907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:20.188260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5ff1-83e8-40c2-b0c9-8465635fee4c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:20.195472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 588/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Fri May 27 09:17:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:22.366062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6c54-1d7d-4115-8832-22b5e70560a3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:22.399715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6c54-1d7d-4115-8832-22b5e70560a3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:22.399715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e6c54-1d7d-4115-8832-22b5e70560a3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:22.399715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 579/1167] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:22.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16b92-b05c-4cbe-bdf1-e9fdf08ed7b5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:22.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16b92-b05c-4cbe-bdf1-e9fdf08ed7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:22.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a16b92-b05c-4cbe-bdf1-e9fdf08ed7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:22.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-14a16b92-b05c-4cbe-bdf1-e9fdf08ed7b5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:22.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-14a16b92-b05c-4cbe-bdf1-e9fdf08ed7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:22.458980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 589/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:22.482280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e6c54-1d7d-4115-8832-22b5e70560a3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:22.534860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:22.582678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e6c54-1d7d-4115-8832-22b5e70560a3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:22.615097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:22.615743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:22.616026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:22.616351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:22.617042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:22.618440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3eb46f-c875-4b9a-85fc-fed996d2e5c8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:22.678500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 580/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Fri May 27 09:17:22 2022] POST /identity/v3/domains => generated 267 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:17:24.905976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874a939-38eb-448a-851e-f58329162305 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:24.906523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874a939-38eb-448a-851e-f58329162305 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:24.906523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874a939-38eb-448a-851e-f58329162305 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:24.907829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 590/1170] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:24.913025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:24.913468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:24.913575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:24.930455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:24.932204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:24.996974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 581/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:25.019597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874a939-38eb-448a-851e-f58329162305 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:25.043798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:25.119068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874a939-38eb-448a-851e-f58329162305 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:25.151970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:25.152809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:25.153083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:25.153351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] PATH_INFO: `/v3/domains/6c8bf81653d548b8aaa3758e2b72e341` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:25.163513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6c8bf81653d548b8aaa3758e2b72e341)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:25.164961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb966d24-48bf-4876-8c6b-22f6bc940c20 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:25.167173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 591/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Fri May 27 09:17:25 2022] GET /identity/v3/domains/6c8bf81653d548b8aaa3758e2b72e341 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:25.173018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:25.214918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:25.268569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82e0cc-c5fc-4328-bb47-220f3f1e289a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:25.302014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:25.302686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:25.302962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:25.303231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:25.303921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:25.305354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75316e-8806-4aa4-9236-c260b5421bc5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:25.393030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 582/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:17:25 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:17:27.613291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eb3e7-201a-4831-baaf-9dfbfa66a85c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:27.613829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eb3e7-201a-4831-baaf-9dfbfa66a85c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:27.613829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eb3e7-201a-4831-baaf-9dfbfa66a85c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:27.615219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 592/1174] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:27.621082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:27.621455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:27.621628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:27.638459 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:27.639623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:27.703457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 583/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:27.727914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eb3e7-201a-4831-baaf-9dfbfa66a85c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:27.810448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:27.828802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eb3e7-201a-4831-baaf-9dfbfa66a85c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:27.862761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:27.863610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:27.863882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:27.864145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] PATH_INFO: `/v3/domains/6c8bf81653d548b8aaa3758e2b72e341` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:27.865294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=6c8bf81653d548b8aaa3758e2b72e341)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:27.866749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8662cb-cc0d-42fe-9bc4-560da6a7f63e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:27.868346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 593/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Fri May 27 09:17:27 2022] GET /identity/v3/domains/6c8bf81653d548b8aaa3758e2b72e341 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:27.874859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:27.898857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:27.972308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08746a3b-ca40-4684-b03d-fe2b7e8592e0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:28.005266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:28.005926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:28.006251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:28.006510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:28.007207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:28.008631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a2673-a383-4c70-9cce-e4820ba75b02 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:28.080507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 584/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:17:27 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:17:30.314002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d244-9c81-4813-9bed-563627f4fa90 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:30.314589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d244-9c81-4813-9bed-563627f4fa90 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:30.314589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d244-9c81-4813-9bed-563627f4fa90 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:30.315920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 594/1178] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:30.321436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:30.321823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:30.321988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:30.339191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:30.340378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:30.404734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 585/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:30.428417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3d244-9c81-4813-9bed-563627f4fa90 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:30.527041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3d244-9c81-4813-9bed-563627f4fa90 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:30.561239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:30.562016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:30.562332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:30.562604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:30.572990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:30.580624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:30.581755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:17:30.582748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 595/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:30.588401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:30.599814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:30.684279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3e5c4-90d2-4a8a-b898-4352ff9862a0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:30.717159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:30.717810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:30.718086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:30.718387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:30.719125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:30.719428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:30.720779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:30.728459 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 586/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:30.734380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:30.746896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:30.831058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e2f42-815d-4a85-9484-47f1754bdf77 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:30.863348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:30.864202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:30.864485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:30.864759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] PATH_INFO: `/v3/users/503f47768dfb4404bb324d610aa8aebd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:30.877473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] RBAC: Authorizing `identity:get_user(user_id=503f47768dfb4404bb324d610aa8aebd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:30.878949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:30.880590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 596/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:30 2022] GET /identity/v3/users/503f47768dfb4404bb324d610aa8aebd => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:30.886890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:30.956538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:30.985155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ca19-e8a8-4484-b9d6-00808327cbe0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:31.018277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:31.018923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:31.019209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:31.019488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:31.085747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:31.089162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:31.155502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:31.185403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 587/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:31.190613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:31.253753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:31.285777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5928d7-e7ee-4dff-b450-13451b8e8f6a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:31.318135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:31.319003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:31.319279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:31.319544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] PATH_INFO: `/v3/projects/35a866b269fb49519c65c15a1278f346` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:31.320946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] RBAC: Authorizing `identity:get_project(project_id=35a866b269fb49519c65c15a1278f346)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:31.322945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf2be3-979e-40fe-979a-643bfae4b265 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:31.324627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 597/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:31 2022] GET /identity/v3/projects/35a866b269fb49519c65c15a1278f346 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:31.330771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:31.425818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c7cf9-04c7-41bc-bae9-66cecf47b738 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:31.459113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:31.459747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:31.460020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:31.460284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:31.461095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=35a866b269fb49519c65c15a1278f346, user.id=503f47768dfb4404bb324d610aa8aebd) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:31.461402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:31.462919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75c077-fb95-4e63-9785-e853fecc4385 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:31.468339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 588/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:17:31 2022] GET /identity/v3/role_assignments?user.id=503f47768dfb4404bb324d610aa8aebd&scope.project.id=35a866b269fb49519c65c15a1278f346&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:33.654765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb893db-73a6-4564-9fce-6ef2c2e589c0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:33.655291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb893db-73a6-4564-9fce-6ef2c2e589c0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:33.655291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb893db-73a6-4564-9fce-6ef2c2e589c0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:33.656673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 598/1186] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:33.662020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:33.662489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:33.662582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:33.680901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:33.682106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:33.711246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:33.747700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 589/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:17:33.770314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb893db-73a6-4564-9fce-6ef2c2e589c0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:33.867995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb893db-73a6-4564-9fce-6ef2c2e589c0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:33.900855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:33.901620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:33.901902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:33.902176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:33.904789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:33.912449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:33.920037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:33.921162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:17:33.922224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 599/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:33.927267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:34.021151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad927dc-df9c-4a46-a1c7-d85e894a1c46 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:34.053140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:34.053789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:34.054063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:34.054358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:34.055087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:34.055389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:34.056771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:34.064567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 590/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:34.070965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:34.166771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30da46-4e21-4a03-9bd2-1bb4f4972d9f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:34.199398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:34.200243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:34.200520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:34.200789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] PATH_INFO: `/v3/users/503f47768dfb4404bb324d610aa8aebd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:34.201943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=503f47768dfb4404bb324d610aa8aebd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:34.203380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:34.204980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 600/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:34 2022] GET /identity/v3/users/503f47768dfb4404bb324d610aa8aebd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:34.210822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:34.240985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:34.311310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7402f3-024e-4583-a1c6-771bbd10deb8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:34.344804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:34.345653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:34.345928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:34.346218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] PATH_INFO: `/v3/projects/35a866b269fb49519c65c15a1278f346` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:34.347379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] RBAC: Authorizing `identity:get_project(project_id=35a866b269fb49519c65c15a1278f346)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:34.348816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00924d63-873c-4deb-ba99-54ffd7b5dd5f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:34.350524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 591/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:34 2022] GET /identity/v3/projects/35a866b269fb49519c65c15a1278f346 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:34.356449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:34.416857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:34.454291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9b1b-c84d-4503-a66b-d347457bc7c3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:34.486472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:34.487345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:34.487625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:34.487896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] PATH_INFO: `/v3/projects/35a866b269fb49519c65c15a1278f346/users/503f47768dfb4404bb324d610aa8aebd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:34.489846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] RBAC: Authorizing `identity:create_grant(project_id=35a866b269fb49519c65c15a1278f346, user_id=503f47768dfb4404bb324d610aa8aebd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:34.491301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cfbdc-f9c3-424c-9b84-9a93b000bb5b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:34.565012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 601/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:17:34 2022] PUT /identity/v3/projects/35a866b269fb49519c65c15a1278f346/users/503f47768dfb4404bb324d610aa8aebd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:17:36.754606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21451c77-aca7-491c-9ad1-19d20b385998 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:36.755160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21451c77-aca7-491c-9ad1-19d20b385998 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:36.755160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21451c77-aca7-491c-9ad1-19d20b385998 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:36.756485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 592/1193] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:17:36.762310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:36.762709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:36.762866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:36.779651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:17:36.780823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:17:36.844379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 602/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:17:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:17:36.867158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21451c77-aca7-491c-9ad1-19d20b385998 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:36.958166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:36.965179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21451c77-aca7-491c-9ad1-19d20b385998 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:36.996692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:36.997460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:36.997735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:36.998003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.007936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:17:37.016869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:17:37.018102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:17:37.019223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 593/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:17:37.024420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:37.077567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:37.120730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25ff45-44ae-49fb-b9e9-0c110a411cd0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:37.153661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:37.154335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:37.154615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:37.154881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.155604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:37.155904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:37.157250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:37.165034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 603/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:37.170975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:37.266880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f98b08-5057-4414-be35-3a541aac8627 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:37.298448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:37.299301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:37.299577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:37.299844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] PATH_INFO: `/v3/users/503f47768dfb4404bb324d610aa8aebd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.300988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] RBAC: Authorizing `identity:get_user(user_id=503f47768dfb4404bb324d610aa8aebd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:37.302440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:37.304024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 594/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Fri May 27 09:17:37 2022] GET /identity/v3/users/503f47768dfb4404bb324d610aa8aebd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:37.310418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:37.406837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4747d3-bf96-44a0-8486-95b2e5fb0f5f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:37.422391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:37.440010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:37.440651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:37.440930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:37.441204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.507645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:37.509182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:37.604973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 604/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:17:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:17:37.610880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:37.636128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:37.708495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4eead3-2199-4db8-bf82-9f6f2ad43dfa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:37.741134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:37.741977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:37.742285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:37.742560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] PATH_INFO: `/v3/projects/35a866b269fb49519c65c15a1278f346` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.744289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] RBAC: Authorizing `identity:get_project(project_id=35a866b269fb49519c65c15a1278f346)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:37.745712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b2fdf-f039-4dc1-9105-aff68df10001 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:37.747433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 595/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Fri May 27 09:17:37 2022] GET /identity/v3/projects/35a866b269fb49519c65c15a1278f346 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:37.753550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:17:37.770450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:37.848798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34d1df-75e7-4c08-8c19-d4fefd885615 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:17:37.882189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:17:37.882851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:17:37.883126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:17:37.883419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:17:37.884249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=35a866b269fb49519c65c15a1278f346, user.id=503f47768dfb4404bb324d610aa8aebd) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:17:37.884556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:17:37.885965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1323bdd-5d74-4895-832e-1e8048c3a0e5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:17:37.891892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 605/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:17:37 2022] GET /identity/v3/role_assignments?user.id=503f47768dfb4404bb324d610aa8aebd&scope.project.id=35a866b269fb49519c65c15a1278f346&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:17:47.737008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=54 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:17:47.889214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=60 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:00.464643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47ff79-a69c-49ee-a34e-964655fa667e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:00.465228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47ff79-a69c-49ee-a34e-964655fa667e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:00.465228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47ff79-a69c-49ee-a34e-964655fa667e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:00.466605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 596/1201] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:00.472039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:00.472482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:00.472579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:00.489996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:00.491229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:00.558263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 606/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:00.583300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47ff79-a69c-49ee-a34e-964655fa667e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:00.688098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47ff79-a69c-49ee-a34e-964655fa667e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:00.722896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:00.723605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:00.723880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:00.724147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:00.733897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:00.739502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:00.752961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:00.754093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:00.755097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:00.755097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:00.755097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:00.755097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:00.755097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 597/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:00.761114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:00.858557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da180c8-f566-4154-b36f-e598921e1d46 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:00.887352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:00.892950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:00.893595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:00.893874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:00.894236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:00.894957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:00.895254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:00.896602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab8612-9fe5-432b-8a4e-48f3bc7ee8bf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:00.907666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 607/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:00.913966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:00.966879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:01.012959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea629114-0aa6-4698-83fe-40ba201de115 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:01.045264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:01.045920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:01.046229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:01.046499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:01.047200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:01.048624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe4bba-9b50-4bcf-b336-a1ba9665276f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:01.213144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 598/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Fri May 27 09:19:00 2022] POST /identity/v3/users => generated 251 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:19:03.458556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f993515-c16c-464d-b314-4bce0914ad8b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:03.459127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f993515-c16c-464d-b314-4bce0914ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:03.459127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f993515-c16c-464d-b314-4bce0914ad8b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:03.460499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 608/1206] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:03.465964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:03.466425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:03.466559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:03.484195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:03.485443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:03.552068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 599/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:03.576754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f993515-c16c-464d-b314-4bce0914ad8b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:03.678922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f993515-c16c-464d-b314-4bce0914ad8b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:03.712367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:03.713080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:03.713361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:03.713637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:03.724162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:03.732647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:03.733731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:03.734650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 609/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:19:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:03.740318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:03.779177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:03.837217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebae64-ecbd-4548-ab28-67f73d01c9f2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:03.870640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:03.871300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:03.871577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:03.871848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:03.872583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:03.872886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:03.874292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:03.882082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 600/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:19:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:03.888018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:03.942333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:03.992042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af1024-b6ef-4b7a-99da-169b10f9e6c3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:04.024409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:04.025186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:04.025469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:04.025739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:04.044301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:04.045744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:04.065852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:04.067028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:04.068316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:04.068316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:04.068316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:04.068316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:04.068316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 610/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:19:04.073374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:04.168902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:04.171711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969899c-9bec-4396-ac0c-2c35575f640d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:04.204834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:04.205488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:04.205768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:04.206044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:04.206803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:04.207106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:04.208499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:04.220169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 601/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:04.226508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:04.324990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa010f1-1001-4023-8d13-6097f54b13ca None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:04.326441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:04.360237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:04.361071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:04.361390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:04.361656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:04.369543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:04.371043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:04.385902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:04.387088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:04.388526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 611/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:04.392947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:04.491273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9d48b-24c8-4e3a-b63d-a5b87d1281f8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:04.525330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:04.525992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:04.526294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:04.526567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:04.527444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:04.531632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:04.531632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:04.547517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 602/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:04.554023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:04.652142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af8ea3-9332-47e6-b38f-f943e37c7296 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:04.685416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:04.686047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:04.686353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:04.686627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:04.715574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:04.752517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:04.753958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:04.856585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 612/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:19:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:19:04.863108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:04.883214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:04.973579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf446a-a3f0-4fad-8349-b7735de789f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:05.009209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:05.010027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:05.010343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:05.010620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:05.021020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:05.023857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:05.041741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:05.043010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:05.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:05.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:05.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:05.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:05.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 603/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:05.048750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:05.109030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:05.146094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10849a20-da4f-49f4-8956-33900de2df29 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:05.180848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:05.181564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:05.181837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:05.182121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:05.182875 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:05.183198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:05.184548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:05.196549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 613/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:05.202351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:05.272948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:05.302834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06381b-022c-4ea4-aae2-58f551d63584 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:05.336982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:05.337775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:05.338050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:05.338349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:05.345278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:05.346760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:05.361565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:05.364864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:05.366029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 604/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:05.368890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:05.468042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88f327-0c78-44c7-8176-f15b471fe085 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:05.490323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:05.501705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:05.502370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:05.502654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:05.502927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:05.503708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:05.504013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:05.505486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42848c-ceb7-44e2-a681-90178ff461b6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:05.521042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 614/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:05.525376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:05.626007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff21ab7-812c-474f-b17b-d5ebe77eadc7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:05.654156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:05.659886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:05.660532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:05.660811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:05.661085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:05.661956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=dc3407db3d994b17851618218efcb7c4) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:05.662298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:05.663717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58054ba-5f03-4379-98b3-19b0486158a0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:05.669013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 605/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:19:05 2022] GET /identity/v3/role_assignments?user.id=dc3407db3d994b17851618218efcb7c4&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:07.833807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17594-f4b6-432c-9217-d3882bdb099e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:07.834874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17594-f4b6-432c-9217-d3882bdb099e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:07.834874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17594-f4b6-432c-9217-d3882bdb099e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:07.836229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 615/1220] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:07.842186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:07.842775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:07.842775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:07.862449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:07.863671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:07.937682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 606/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:07.961291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17594-f4b6-432c-9217-d3882bdb099e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.064726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17594-f4b6-432c-9217-d3882bdb099e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.099203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.099833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.100110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.100378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.110383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:08.118490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:08.124895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:08.125776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 616/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:19:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:08.126521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.230859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1e9ec-a5f2-44e7-b2ba-cb076047a7b9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.254499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:08.264934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.265601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.265876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.266147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.266923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:08.267224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:08.268587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:08.277116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 607/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:08.283087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.331628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:08.396936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e9894-b153-45e4-81dc-0d71a74864fc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.430870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.431657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.431934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.432205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.443689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:08.445708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:08.463717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:08.464821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:08.466116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:08.466116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:08.466116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:08.466116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:08.466116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 617/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:08.470766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.576279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192990c1-22b0-434c-8a31-d7e274041568 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.611933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.612638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.612924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.613200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.613925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:08.614272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:08.616174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:08.629336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 608/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:08.635949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.730054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:08.736739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24ba1b-0bab-49ce-a7ae-28ca016f0e9e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.771196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.771981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.772262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.772534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.780002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:08.781563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:08.795940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:08.797033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:08.797970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 618/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:08.803040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:08.851760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:08.906763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a982-0cde-4119-a79d-a393e38f12f7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:08.943483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:08.944716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:08.945003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:08.945322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:08.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:08.946511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:08.947961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:08.965996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 609/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:08.971746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:09.073329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06e536-ef70-4cd6-bea8-a3cb1950be95 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:09.107057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:09.107843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:09.108123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:09.108397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:09.119599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:09.121050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:09.137901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:09.140947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:09.142230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:09.142230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:09.142230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:09.142230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:09.142230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 619/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:09.144889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:09.244555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba7-b901-4449-a073-cf1559993772 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:09.246829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:09.281360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:09.282025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:09.282342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:09.282619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:09.283340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:09.283647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:09.285033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:09.297050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 610/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:09.302936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:09.312793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:09.403988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d84a34-b558-42ed-b77a-81f8a3fc6ae6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:09.438911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:09.439700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:09.439977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:09.440244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:09.447249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:09.448693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:09.463514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:09.464579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:09.465626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 620/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:09.470708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:09.573983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa314-035d-48e9-8bba-433a9eda820f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:09.608720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:09.609376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:09.609657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:09.609936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:09.610760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:09.611069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:09.612540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b265676-74bc-40f0-8c28-be0965f44151 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:09.625208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 611/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:09.631415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:09.697515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:09.733019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76975adf-6676-47f5-9541-840ed6d57cdf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:09.766445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:09.767322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:09.767609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:09.767885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/dc3407db3d994b17851618218efcb7c4/roles/fd8c102f47b04674a41cc4a0f66f1cad` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:09.782511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=dc3407db3d994b17851618218efcb7c4, role_id=fd8c102f47b04674a41cc4a0f66f1cad)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:09.783956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60097414-d19b-4835-9bab-81c417202fa6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:09.857579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 621/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:19:09 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/dc3407db3d994b17851618218efcb7c4/roles/fd8c102f47b04674a41cc4a0f66f1cad => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:19:12.048218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186188f8-591b-43f3-a62c-89058e9bcda0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.048837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186188f8-591b-43f3-a62c-89058e9bcda0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.048837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186188f8-591b-43f3-a62c-89058e9bcda0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.050160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 612/1233] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:12.055577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.056132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.056132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.073404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:12.074627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:12.142646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 622/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:12.166507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186188f8-591b-43f3-a62c-89058e9bcda0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:12.191677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:12.268057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186188f8-591b-43f3-a62c-89058e9bcda0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:12.305262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:12.305977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.306287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.306566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.320713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:12.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:12.329933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:12.331217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 613/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Fri May 27 09:19:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:12.338577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:12.390811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:12.447733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5371716-06fd-440a-a7f9-107b2e885dbc None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:12.480817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:12.481475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.481755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.482024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.482799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:12.483106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:12.484470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:12.493615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 623/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Fri May 27 09:19:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:12.499964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:12.588763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:12.604740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e681b2-a626-491a-bbcd-4dde2247434c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:12.639708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:12.640503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.640781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.641053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.651633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:12.653095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:12.670496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:12.671589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:12.672804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:12.672804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:12.672804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:12.672804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:12.672804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 614/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:12.677469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:12.778072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a531-5626-4f61-904f-2f2d8817e30b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:12.787839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:12.812799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:12.813454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.813735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.814010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.814759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:12.815063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:12.816433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:12.828061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 624/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:12.833781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:12.934979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e43b-64d5-4cbe-b2e8-f9c02a846571 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:12.968849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:12.969684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:12.969973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:12.970272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:12.973058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:12.978267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:12.979716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:12.994828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:12.996155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:12.997476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 615/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:13.001890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:13.104548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f67d3-e031-4d75-833e-491287eef434 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:13.138579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:13.139231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:13.139505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:13.139770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:13.140589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:13.140895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:13.142367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:13.159075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 625/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:13.164881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:13.263497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bd9c-233c-4539-9733-16d1fae97e60 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:13.298934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:13.299585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:13.299923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:13.300291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:13.362835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:13.370640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:13.372133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:13.473059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 616/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:19:13.480431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:13.508946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:13.582079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f35b6-688b-4f44-9b8a-7c4efe024c04 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:13.616572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:13.617283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:13.617558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:13.617828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:13.628073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:13.629529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:13.646728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:13.647941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:13.649065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:13.649065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:13.649065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:13.649065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:13.649065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 626/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:13.653770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:13.749030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:13.754828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed6d55-bbde-4817-b414-cbbea74e081a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:13.788900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:13.789555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:13.789840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:13.790113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:13.790854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:13.791157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:13.792582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:13.803942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 617/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:13.809887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:13.888212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:13.909257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352e988-6941-452c-a2e0-151a440b588b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:13.944366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:13.945323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:13.945480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:13.945676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:13.952633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:13.954106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:13.969179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:13.970407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:13.971355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 627/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:13.977040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:14.086739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69da67-eb1e-46e1-ad33-8c2d8dc501c8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:14.119982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:14.120628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:14.120905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:14.121197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:14.121943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:14.122272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:14.123700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e12315-6207-4c41-a0b9-84a1b09eba10 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:14.135815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 618/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:14.142312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:14.245762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64898a6-3a12-4103-9a96-a79bab5c5cf4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:14.282039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:14.282761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:14.283042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:14.283316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:14.284203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] RBAC: Adding query filter params (role.id=fd8c102f47b04674a41cc4a0f66f1cad, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=dc3407db3d994b17851618218efcb7c4) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:14.284518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:14.285948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c35a9-1c09-4d94-9c07-e7cdf54fca89 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:14.291949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 628/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:19:14 2022] GET /identity/v3/role_assignments?user.id=dc3407db3d994b17851618218efcb7c4&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=fd8c102f47b04674a41cc4a0f66f1cad => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:16.556769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0418-9800-4cee-bfee-828952683297 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:16.557339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0418-9800-4cee-bfee-828952683297 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:16.557339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0418-9800-4cee-bfee-828952683297 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:16.558683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 619/1247] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:16.564687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:16.565247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:16.565247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:16.571714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:16.585740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:16.587071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:16.661286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 629/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:16.685380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b0418-9800-4cee-bfee-828952683297 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:16.705414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:16.790440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b0418-9800-4cee-bfee-828952683297 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:16.824021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:16.824718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:16.824990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:16.825536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:16.837514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:16.844990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:16.846058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:16.847217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 620/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:16.852983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:16.950940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ecacf-8f4e-4802-9763-8f1c2929a66b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:16.981322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:16.986331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:16.986991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:16.987269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:16.987539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:16.988279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:16.988594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:16.989946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:16.998457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 630/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:17.004199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:17.081421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:17.104460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592528f5-9b56-4c6a-ae67-989f7fb1b9a3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:17.136301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:17.137075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:17.137352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:17.137619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:17.148308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:17.149752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:17.166397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:17.167318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:17.168246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:17.168246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:17.168246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:17.168246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:17.168246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 621/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:17.173565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:17.276700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528df14-214b-4d21-8fef-48237cec07c5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:17.310599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:17.311256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:17.311539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:17.311815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:17.312526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:17.312882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:17.314282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:17.326305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 631/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:17.332271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:17.432179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fedec6-24a3-45ca-b78f-17d98cd37f83 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:17.457321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:17.465069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:17.465854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:17.466136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:17.466433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:17.473764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:17.475765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:17.490063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:17.491603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:17.492619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 622/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:17.497755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:17.553733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:17.597700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4cb0-e8d0-413c-a917-230dbcc3f836 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:17.630689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:17.631416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:17.631977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:17.631977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:17.632793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:17.633106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:17.634620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:17.654501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 632/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:17.660458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:17.762859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a93ce-c997-416e-9e3d-fe81b49f1bd3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:17.797177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:17.797871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:17.798133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:17.798455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:17.853585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:17.865274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:17.867304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:17.963739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 623/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:19:17.969656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:18.070223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9934-8e16-4d9b-80f2-42962514ab49 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:18.084219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:18.105760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:18.106553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:18.112277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:18.112277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:18.119361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:18.120821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:18.138361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:18.139482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:18.140615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:18.140615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:18.140615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:18.140615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:18.140615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 633/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:18.145633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:18.234709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:18.246779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae54c8-e693-40f1-a923-bc919553ef9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:18.279852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:18.280522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:18.280803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:18.281078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:18.281800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:18.282108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:18.283520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:18.295212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 624/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:18.301000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:18.402988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dc324-28de-4d91-8a25-07c8f9059fad None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:18.437641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:18.438477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:18.438766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:18.439041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:18.446885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:18.448347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:18.462951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:18.464046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:18.465182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 634/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:18.470298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:18.572742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ce19b-5c2d-499d-95f1-b25255bf301a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:18.610659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:18.611319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:18.611601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:18.611870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:18.612647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:18.612954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:18.614402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11f076-7da3-4788-b7b7-ead68f77ccfd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:18.626510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 625/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:18.632712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:18.660461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:18.733659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c03901-aa2b-4f83-844b-ffe5267c4a32 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:18.767629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:18.768278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:18.768560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:18.768836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:18.769664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=dc3407db3d994b17851618218efcb7c4) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:18.769976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:18.771437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b3a1c-d4a4-4ff8-a5d5-a6ea28c47a42 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:18.777258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 635/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:19:18 2022] GET /identity/v3/role_assignments?user.id=dc3407db3d994b17851618218efcb7c4&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:20.988356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a7dd-9c41-4288-82ac-4ec83364cc6e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:20.988863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a7dd-9c41-4288-82ac-4ec83364cc6e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:20.988863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a7dd-9c41-4288-82ac-4ec83364cc6e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:20.990177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 626/1261] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:20.996284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:20.996774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:20.996887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.013771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:21.014989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:21.083403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 636/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 09:19:21.107546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330a7dd-9c41-4288-82ac-4ec83364cc6e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:21.119815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:21.215821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330a7dd-9c41-4288-82ac-4ec83364cc6e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:21.251676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:21.252473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:21.252756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:21.253740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.265327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:21.273822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:21.281482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:21.282576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 627/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:19:21.283450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:21.300054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:21.384840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f96db-90b6-4c3d-9699-71384bc157a4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:21.419778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:21.420444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:21.420725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:21.421005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.421746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:21.422054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:21.423461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:21.434570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 637/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:21.438458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:21.510452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:21.544385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d10c93-40c9-4b27-86fc-c5f9002d8ff1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:21.579109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:21.579901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:21.580184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:21.580458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.590974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:21.592438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:21.610008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:21.611130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:21.612224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:21.612224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:21.612224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:21.612224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:21.612224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 628/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:21.617779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:21.698782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:21.716830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff4417-d057-4dd5-8687-b79113425440 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:21.749480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:21.750126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:21.750451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:21.750721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.751521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:21.751723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:21.753076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:21.764687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 638/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:21.770301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:21.869334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5cfc5-3ff5-42a1-843f-4f34991ba2a1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:21.888086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:21.901600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:21.902486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:21.902778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:21.907673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:21.910945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:21.912935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:21.927530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:21.929238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:21.930561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 629/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:21.935443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.032633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec66f19-bcab-4791-b0ec-4073666e6b94 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.065716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.066388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.066666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.066932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.068003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:22.068326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.069709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.078092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:22.087197 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 639/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:22.093097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.193209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bb798-517d-485a-8884-0f49420dc70c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.228250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.229049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.229332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.229608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.241155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.242650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:22.259873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:22.260922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:22.262057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:22.262057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:22.262057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:22.262057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:22.262057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 630/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:22.268724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.370274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109587f-df59-4abf-93fb-cdb7152388ec None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.403693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.404420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.404693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.404959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.405669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:22.405968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.407367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.418708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 640/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:22.424436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.456680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:22.525753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c66d-9bbd-4133-a2b9-c6a95ed7cdb7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.558958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.559736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.560016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.560283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.566896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.570689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:22.582851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:22.583888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:22.584977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 631/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:22.589918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.649777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:22.691987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d65c6-c392-4e0c-bc80-0b3080490a43 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.725404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.726076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.726380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.726646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.727481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:22.727793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.729207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa04af0-119d-4e25-9ccd-8ae503d82506 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.741964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 641/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:22.748033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:22.844093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:22.853230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e78604-f0fb-4a5c-af51-9878bfcdbe09 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:22.885890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:22.886834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:22.887110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:22.887378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] PATH_INFO: `/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/dc3407db3d994b17851618218efcb7c4/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:22.889277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=010e30be1e8a43ceaa3bede3ac4c9a90, user_id=dc3407db3d994b17851618218efcb7c4, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:22.890747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95579e8f-b866-4e70-9398-3351fd31d3c1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:22.977878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 632/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Fri May 27 09:19:22 2022] PUT /identity/v3/projects/010e30be1e8a43ceaa3bede3ac4c9a90/users/dc3407db3d994b17851618218efcb7c4/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:19:25.167244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504c5d-fd6d-4080-a76b-720d2ea420f9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.167812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504c5d-fd6d-4080-a76b-720d2ea420f9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.167902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504c5d-fd6d-4080-a76b-720d2ea420f9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.169371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 642/1274] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:25.174778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.175267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.175385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.192685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:25.194483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:25.260673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 633/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:25.284508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504c5d-fd6d-4080-a76b-720d2ea420f9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:25.391647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504c5d-fd6d-4080-a76b-720d2ea420f9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:25.409595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:25.426521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:25.427282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.427558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.428366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.438606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 09:19:25.447171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:25.448460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:25.449422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 643/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Fri May 27 09:19:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:25.454704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:25.537862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:25.557654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb08391-e27d-473b-ae00-d59b3b1cf992 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:25.593848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:25.598892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:25.608041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 634/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Fri May 27 09:19:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:25.613882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:25.723668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ab24-8559-428b-9cf1-925418a5c553 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:25.765354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:25.766142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.766449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.766723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.777710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:25.779196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:25.796693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:25.798916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:25.800086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:25.800086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:25.800086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:25.800086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:25.800086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 644/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:25.804552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:25.904887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2d347-4dbd-4e5a-a41a-a3b47f37bf20 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:25.938124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:25.938810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:25.939087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:25.939357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:25.940071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:25.940374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:25.941743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:25.954056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 635/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:25.960294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:25.994994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:26.059162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ddf9-1673-412b-ba53-4f8219b4a1eb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:26.092647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:26.093433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:26.093712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:26.093985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:26.103239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:26.105355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 09:19:26.119798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:26.121069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:26.122325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 645/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:26.127294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:26.131135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:26.228240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a03b-331a-4f13-bb86-02699a726d20 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:26.261818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:26.262517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:26.262796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:26.263061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:26.263886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:26.264192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:26.265695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:26.284248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 636/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Fri May 27 09:19:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:26.289952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:26.381792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:26.391224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141d0d-ce3c-44b2-bdd7-67763c2a1e47 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:26.423608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:26.424238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:26.424525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:26.424798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:26.492930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:26.494486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:26.586843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:26.595308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 646/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Fri May 27 09:19:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 09:19:26.601296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:26.663730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:26.704491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53492e49-138d-4ed9-8de7-00fae73d5ce5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:26.737014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:26.737795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:26.738076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:26.738380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:26.750075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:26.751568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 09:19:26.769096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:26.770268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:26.771402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 09:19:26.771402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 09:19:26.771402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 09:19:26.771402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:26.771402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 637/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:26.776110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:26.873280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb3696-689d-4404-aeb5-885de28df19d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:26.909496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:26.910683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:26.911210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:26.911722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:26.913728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:26.914043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:26.915453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:26.927531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 647/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Fri May 27 09:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:26.933396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:27.035327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81938e-9b3f-4df6-9112-e96a5c7f9e19 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:27.049269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:27.070274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:27.071062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:27.071340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:27.071610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:27.078546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:27.080022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 09:19:27.094558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:27.095603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application May 27 09:19:27.096550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 638/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Fri May 27 09:19:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 09:19:27.101885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:27.154773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:27.203576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce396b-8b32-4d2f-afbb-168cfdb90bf3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:27.236510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:27.237172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:27.237452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:27.237726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:27.241528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:27.241528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:27.241528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ed89-cd80-47f3-9970-8d06b668fda6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:27.253203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 648/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Fri May 27 09:19:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:27.259831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:27.360813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757355b-c673-41f8-b234-85e8fb527b9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:27.394501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:27.395145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:27.397858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:27.397858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:27.397858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] RBAC: Adding query filter params (role.id=2cf0ecb1f1344e92aec134e29d0777e2, scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90, user.id=dc3407db3d994b17851618218efcb7c4) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:27.397858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:27.398306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03f6dc-3d1c-4479-9644-1d149433c1fd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:27.404364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 639/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Fri May 27 09:19:27 2022] GET /identity/v3/role_assignments?user.id=dc3407db3d994b17851618218efcb7c4&scope.project.id=010e30be1e8a43ceaa3bede3ac4c9a90&role.id=2cf0ecb1f1344e92aec134e29d0777e2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:29.670013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462f7e-1cde-4f25-a460-b82526811859 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:29.670635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462f7e-1cde-4f25-a460-b82526811859 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:29.670745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462f7e-1cde-4f25-a460-b82526811859 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:29.672056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 649/1288] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:29.677506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:29.677929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:29.678071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:29.695252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:29.696447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:29.727660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:29.763193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 640/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:29.785564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462f7e-1cde-4f25-a460-b82526811859 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:29.888794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462f7e-1cde-4f25-a460-b82526811859 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:29.903351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:29.929132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:29.929934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:29.930241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:29.930514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:29.931174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:29.932587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:29.946165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:29.947524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:29.948383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 650/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:19:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:29.953866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:30.068947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b98c9d-d30b-428b-92fc-66570a06cede None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:30.106315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:30.106977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:30.107256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:30.107527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:30.108245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:30.108531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:30.109917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aca8b-1831-490d-b6c6-e8284e96a9e9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:30.114813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 641/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Fri May 27 09:19:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:30.120520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:30.229796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52199-8306-41a1-bcc4-c728d19a342f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:30.263919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:30.264584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:30.264857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:30.266916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:30.266916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:30.266916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:30.267655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281aa04-2fc9-4d8b-9ceb-88639e9b9163 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:30.273134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 651/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Fri May 27 09:19:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:32.471333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca731-4918-4e74-8386-5bc46017dd35 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:32.472309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca731-4918-4e74-8386-5bc46017dd35 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:32.472309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca731-4918-4e74-8386-5bc46017dd35 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:32.473357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 642/1293] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:32.479434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9cb34-3cb5-43d9-955a-f4b30c105937 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:32.479879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9cb34-3cb5-43d9-955a-f4b30c105937 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:32.479994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9cb34-3cb5-43d9-955a-f4b30c105937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:32.497640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-25d9cb34-3cb5-43d9-955a-f4b30c105937 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:32.498873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-25d9cb34-3cb5-43d9-955a-f4b30c105937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:32.528186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:32.565866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 652/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:32.589353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca731-4918-4e74-8386-5bc46017dd35 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:32.622595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:32.698591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca731-4918-4e74-8386-5bc46017dd35 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:32.735292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:32.735824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:32.736089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:32.736360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:32.737093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:32.738540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02346983-3ab2-4297-8488-fe44389c68f1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:32.823498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 643/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Fri May 27 09:19:32 2022] POST /identity/v3/services => generated 246 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:19:35.073432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09ced-5a1f-45dc-826e-7bbecc282d2d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.073999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09ced-5a1f-45dc-826e-7bbecc282d2d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.073999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09ced-5a1f-45dc-826e-7bbecc282d2d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.075529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 653/1296] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:35.080886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.081379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.081618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.099038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:35.100223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:35.173494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 644/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:35.197347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f09ced-5a1f-45dc-826e-7bbecc282d2d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:35.303874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f09ced-5a1f-45dc-826e-7bbecc282d2d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:35.336743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:35.337454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.337738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.338013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.338698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:35.340068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:35.352516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:35.353722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:35.354633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 654/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:19:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:35.360103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:35.378725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:35.457306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6ff2f-41bd-4f9f-8448-59951a3cc5ba None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:35.492120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:35.492766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.493149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.493421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.494114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:35.494443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:35.495800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:35.501009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 645/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Fri May 27 09:19:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:19:35.507075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:35.539530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:35.609027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe226e7-9423-4d09-9bf8-fb9b66622c77 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:35.641100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:35.641747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.642027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.642335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.643117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] RBAC: Adding query filter params (interface=public, service_id=6f8552e5fb47400fa06b8e2b76ebec69, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:35.643469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:35.644782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb05c-b665-4bb1-adfb-9a846ab2ed0a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:35.650316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 655/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Fri May 27 09:19:35 2022] GET /identity/v3/endpoints?service_id=6f8552e5fb47400fa06b8e2b76ebec69&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:35.656081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:35.739285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:35.756927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f1c2-f9a1-49d0-b0fc-a43482af747e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:35.791368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:35.792021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:35.792304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:35.792577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:35.793244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:35.794734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f96d25-82c8-4083-98fc-cf5e5cfc6fce None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:35.802149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 646/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Fri May 27 09:19:35 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:19:38.001915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bafc-a81f-4cbc-bdb9-15f710b1f7b2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:38.002511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bafc-a81f-4cbc-bdb9-15f710b1f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:38.002511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bafc-a81f-4cbc-bdb9-15f710b1f7b2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:38.003849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 656/1302] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:19:38.009802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:38.010297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:38.010390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:38.026466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:19:38.027638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:19:38.093173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 647/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:19:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 09:19:38.117415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9bafc-a81f-4cbc-bdb9-15f710b1f7b2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:38.215499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:38.221839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9bafc-a81f-4cbc-bdb9-15f710b1f7b2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:38.257298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:38.258157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:38.258474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:38.258752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:38.259404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:38.260803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 09:19:38.274431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 09:19:38.276153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application May 27 09:19:38.277006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 657/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Fri May 27 09:19:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 09:19:38.282252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:38.360799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:38.383995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a35d68-8bb2-43ad-8493-52b5f4133604 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:38.418412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:38.419076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:38.419358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:38.419653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:38.420337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:19:38.420752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:38.422174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5322581-2408-413f-93f5-5c4e8fb82ec2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:38.427318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 648/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Fri May 27 09:19:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:19:38.433128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:19:38.537595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef379-8c86-4b54-8aad-baf0fce194dd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:19:38.572064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:19:38.572713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:19:38.572990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:19:38.573259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:19:38.573998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:19:38.575425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4b99-bbef-46e9-bc97-490d376c437d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:19:38.692032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 658/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Fri May 27 09:19:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:19:48.439244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=30 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:19:48.686664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:21.794340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad111-7afa-4632-9a28-1ce42505505d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:21.794340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad111-7afa-4632-9a28-1ce42505505d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:21.794340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad111-7afa-4632-9a28-1ce42505505d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:21.794340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 649/1307] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:21.795206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ceed-4039-4c19-a02c-14981e2a238a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:21.795206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ceed-4039-4c19-a02c-14981e2a238a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:21.795206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203ceed-4039-4c19-a02c-14981e2a238a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:21.906368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a203ceed-4039-4c19-a02c-14981e2a238a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:21.907894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a203ceed-4039-4c19-a02c-14981e2a238a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:22.044521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 659/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:22.102828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:22.103164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:22.103377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:22.105079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 650/1309] 10.222.0.48 () {54 vars in 964 bytes} [Fri May 27 09:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:22.111706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8838f5-9ed5-4d24-b706-6316218bcb37 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:22.112325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8838f5-9ed5-4d24-b706-6316218bcb37 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:22.112607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8838f5-9ed5-4d24-b706-6316218bcb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:22.231461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cc8838f5-9ed5-4d24-b706-6316218bcb37 None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:22.232822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cc8838f5-9ed5-4d24-b706-6316218bcb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:22.257839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:22.324106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 660/1310] 10.222.0.48 () {58 vars in 1061 bytes} [Fri May 27 09:27:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:22.334305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:22.336255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:22.482557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7f886-3837-49d0-bed1-3da994232bcd None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:22.908195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:22.908195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:22.908195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:22.908195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:22.909844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:22.909844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:22.909844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:22.909844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1bbea-d414-4d92-9ad0-0fc007d37868 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:22.909844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 651/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:25.498413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:25.499265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:25.499451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:25.500840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 661/1312] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:25.506501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7613-67c7-4332-aa36-83fd2de3bbf9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:25.506954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7613-67c7-4332-aa36-83fd2de3bbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:25.507238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7613-67c7-4332-aa36-83fd2de3bbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:25.599889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6a2e7613-67c7-4332-aa36-83fd2de3bbf9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:25.601939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6a2e7613-67c7-4332-aa36-83fd2de3bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:25.767963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 652/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:25.828223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:25.829851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:25.970124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd809d67-e4d6-4ab8-8c68-fa996dae0e0b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:26.075540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:26.079468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:26.080381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:26.080793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:26.081190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:26.083925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:26.246327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:26.248310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf177d9-d134-44bf-8a48-dc0a8bf01399 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:26.276345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 662/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:29.194299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd5c5-e1f8-4c19-bf7f-90c0c4a8eb78 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:29.194824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd5c5-e1f8-4c19-bf7f-90c0c4a8eb78 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:29.194824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd5c5-e1f8-4c19-bf7f-90c0c4a8eb78 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:29.196569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 653/1315] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:29.202719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253df216-ce01-4778-8eaf-f787161dd681 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:29.203345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253df216-ce01-4778-8eaf-f787161dd681 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:29.203345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253df216-ce01-4778-8eaf-f787161dd681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:29.292688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-253df216-ce01-4778-8eaf-f787161dd681 None None] MFA Rules not processed for user `f83fca0dd8e84a8191f04222fdc03da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:29.294108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-253df216-ce01-4778-8eaf-f787161dd681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:29.361951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:29.416258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 663/1316] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:29.474330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:29.475073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:29.475489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:29.477344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 654/1317] 10.222.0.48 () {54 vars in 964 bytes} [Fri May 27 09:27:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:29.484382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f685-3a6a-461a-97db-fc0719d72bcc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:29.484858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f685-3a6a-461a-97db-fc0719d72bcc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:29.485273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5f685-3a6a-461a-97db-fc0719d72bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:29.596036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bbb5f685-3a6a-461a-97db-fc0719d72bcc None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:29.597358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bbb5f685-3a6a-461a-97db-fc0719d72bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:29.699719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 664/1318] 10.222.0.48 () {58 vars in 1061 bytes} [Fri May 27 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:27:29.710467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:29.713015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:29.729629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:29.845356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873b4cd-e1cc-4e5a-876d-31f97465fe31 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:29.931452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:29.932094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:29.932367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:29.932626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:29.934375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:30.048879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:30.120794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:30.123510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eae0d3-df6d-4a9a-ab9b-24fa7d38c8d0 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:30.155974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 655/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:33.674565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:33.675141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:33.675141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:33.676464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 665/1320] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:33.681785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5e0b-c161-4e2f-bbb8-7957375d9fec None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:33.682234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5e0b-c161-4e2f-bbb8-7957375d9fec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:33.682351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5e0b-c161-4e2f-bbb8-7957375d9fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:33.753073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e4ed5e0b-c161-4e2f-bbb8-7957375d9fec None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:33.754334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e4ed5e0b-c161-4e2f-bbb8-7957375d9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:33.852775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 656/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:33.892947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:33.894511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:33.947221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:34.001034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0199-cd66-4c03-8562-b5dfc2f5271d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:34.078523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:34.079171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:34.079479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:34.079749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:34.081410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:34.180837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:34.195560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:34.198181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d87e37-3779-4111-94ad-b1625fcf5610 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:34.217485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 666/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:37.686010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:37.686838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:37.686838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:37.688197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 657/1323] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:37.694084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489d45-9e95-46fc-bfc4-d4ea865ccb73 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:37.694539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489d45-9e95-46fc-bfc4-d4ea865ccb73 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:37.694651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489d45-9e95-46fc-bfc4-d4ea865ccb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:37.760236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-06489d45-9e95-46fc-bfc4-d4ea865ccb73 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:37.761446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-06489d45-9e95-46fc-bfc4-d4ea865ccb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:37.865480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 667/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:27:37.901435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:37.903101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:37.924143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:38.011759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fce840-5ab6-40a5-a803-852e9333ed13 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:38.090024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:38.090710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:38.090995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:38.091265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:38.092941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:38.168677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:38.209521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:38.211006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4628b3-dfdb-4f90-98ce-ec6693be3378 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:38.230429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 658/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:40.545480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:40.546015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:40.546015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:40.547432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 668/1326] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:40.552808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd594b-79a9-4176-9a0e-3f27b57dfe12 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:40.553234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd594b-79a9-4176-9a0e-3f27b57dfe12 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:40.553359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd594b-79a9-4176-9a0e-3f27b57dfe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:40.622167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1cdd594b-79a9-4176-9a0e-3f27b57dfe12 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:40.623398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1cdd594b-79a9-4176-9a0e-3f27b57dfe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:40.722776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 659/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:40.758912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:40.760443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:40.822796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:40.867632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fbe25-4afa-47ce-b063-cebe9a93d657 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:40.941177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:40.941830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:40.942641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:40.942922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:40.944740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:41.052854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:41.055274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:41.056711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33952b-f77d-4322-b151-d9e7d31178ad service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:41.075535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 669/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:43.416505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:43.417080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:43.417222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:43.418647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 660/1329] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:43.424637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f9bcc-404c-48dc-8eb5-3503279cc3d9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:43.425132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f9bcc-404c-48dc-8eb5-3503279cc3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:43.425252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f9bcc-404c-48dc-8eb5-3503279cc3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:43.502785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7e8f9bcc-404c-48dc-8eb5-3503279cc3d9 None None] MFA Rules not processed for user `f83fca0dd8e84a8191f04222fdc03da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:43.503990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7e8f9bcc-404c-48dc-8eb5-3503279cc3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:43.614655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 670/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:43.652309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:43.653984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:43.688688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:43.771428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaf509-ffae-4df8-8a3c-39018dc9e623 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:43.853590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:43.854273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:43.854559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:43.854833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:43.856534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:43.948324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:43.983287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:43.984739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15b030-e3c5-4d93-bcfc-7d60b20f41c4 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:44.006408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 661/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:46.759348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:46.759905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:46.759905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:46.761277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 671/1332] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:46.767125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45281db-5f9e-49ba-a024-99067f11a822 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:46.767580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45281db-5f9e-49ba-a024-99067f11a822 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:46.767670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45281db-5f9e-49ba-a024-99067f11a822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:46.836333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f45281db-5f9e-49ba-a024-99067f11a822 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:46.837560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f45281db-5f9e-49ba-a024-99067f11a822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:46.943272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 662/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:27:46.979665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:46.981202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:47.055540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:47.095915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a67b8-3cac-4310-a26c-9819b0027aab None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:47.180133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:47.180788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:47.181068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:47.181325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:47.183172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:47.305446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:47.306966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93940097-bf5c-4452-ae24-9b6e51349da5 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:47.323844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:47.328741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 672/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:49.682641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:49.683187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:49.683187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:49.684609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 663/1335] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:49.690634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29e28-6133-4cda-9fdb-739635532be9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:49.691188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29e28-6133-4cda-9fdb-739635532be9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:49.691188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29e28-6133-4cda-9fdb-739635532be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:49.760961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6ed29e28-6133-4cda-9fdb-739635532be9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:49.761836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6ed29e28-6133-4cda-9fdb-739635532be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:49.862344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 673/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:49.901846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:49.903427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:49.945142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:50.011006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db5e53-00f0-4876-8ced-59dbbbd74750 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:50.092960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:50.093602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:50.093877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:50.094147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:50.095834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:50.181439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:50.207185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:50.209239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d89c7-2e20-4929-99a6-1429e214141d service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:50.227219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 664/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:53.088778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:53.089352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:53.089352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:53.090892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 674/1338] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:53.096588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6462d48-faac-4d7b-8f90-a61650e90f17 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:53.097134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6462d48-faac-4d7b-8f90-a61650e90f17 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:53.097134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6462d48-faac-4d7b-8f90-a61650e90f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:53.179180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d6462d48-faac-4d7b-8f90-a61650e90f17 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:53.180416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d6462d48-faac-4d7b-8f90-a61650e90f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:53.297195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 665/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:53.334087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:53.339290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:53.423024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:27:53.439063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284923-2a5c-438c-abe0-79c9d944c118 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:53.443374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:53.444016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:53.444292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:53.444560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:53.454407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:53.455835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563989c5-c1de-4994-bd3d-d39454a654ac service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:53.465439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 675/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:27:58.066754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:58.067354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:58.067354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:58.068891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 666/1341] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:27:58.074728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e90e-6d85-4696-8361-1a16371e8a4c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:58.074728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e90e-6d85-4696-8361-1a16371e8a4c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:58.074728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1e90e-6d85-4696-8361-1a16371e8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:58.112146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c3e1e90e-6d85-4696-8361-1a16371e8a4c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:27:58.113606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c3e1e90e-6d85-4696-8361-1a16371e8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:58.200000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 676/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:27:58.240313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:27:58.242301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:27:58.256074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680c7a6-4793-4786-bb45-0911096348f4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:27:58.259596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:27:58.260261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:27:58.260539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:27:58.260819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:27:58.272568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:27:58.274009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cf6a5-474d-4e60-b865-e6947ee38d8f service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:27:58.284046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 667/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:01.797383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa1260-1eb6-4335-9ee6-262d8240dcee None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:01.797918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa1260-1eb6-4335-9ee6-262d8240dcee None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:01.797918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa1260-1eb6-4335-9ee6-262d8240dcee None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:01.799346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 677/1344] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:01.804767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cde9-5359-4b94-a9de-8c813a3a6abb None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:01.805247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cde9-5359-4b94-a9de-8c813a3a6abb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:01.805350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03cde9-5359-4b94-a9de-8c813a3a6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:01.823494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8b03cde9-5359-4b94-a9de-8c813a3a6abb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:01.824709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8b03cde9-5359-4b94-a9de-8c813a3a6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:01.833122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 668/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:01.870217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa1260-1eb6-4335-9ee6-262d8240dcee None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:01.881682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa1260-1eb6-4335-9ee6-262d8240dcee None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:01.885164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:01.885792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:01.886076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:01.886380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:01.896652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:01.898077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc7838-1645-4a3a-a1fd-e495fa282af0 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:01.907948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 678/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:28:07.000562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9629cd-cfd2-47fd-bd37-275b00a8eed2 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:07.001108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9629cd-cfd2-47fd-bd37-275b00a8eed2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:07.001226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9629cd-cfd2-47fd-bd37-275b00a8eed2 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:07.002808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 669/1347] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:07.009408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf471d7-163b-4dce-89f3-6f3132ae331d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:07.009935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf471d7-163b-4dce-89f3-6f3132ae331d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:07.009935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf471d7-163b-4dce-89f3-6f3132ae331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:07.029188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-edf471d7-163b-4dce-89f3-6f3132ae331d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:07.030465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-edf471d7-163b-4dce-89f3-6f3132ae331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:07.038723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 679/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:28:07.077081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9629cd-cfd2-47fd-bd37-275b00a8eed2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:07.088610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9629cd-cfd2-47fd-bd37-275b00a8eed2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:07.092097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:07.092730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:07.093010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:07.093283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:07.103926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:07.105395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065084-b34e-41fa-8a10-04056dce5db5 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:07.118263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 670/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:12.673724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d0fe-c7ab-47bf-8b82-f95c9dd6e9a3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:12.674347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d0fe-c7ab-47bf-8b82-f95c9dd6e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:12.674347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d0fe-c7ab-47bf-8b82-f95c9dd6e9a3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:12.677233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 680/1350] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:12.682885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5626-5847-421d-86e4-c7fa0b553335 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:12.683852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5626-5847-421d-86e4-c7fa0b553335 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:12.684404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f5626-5847-421d-86e4-c7fa0b553335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:12.719250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:12.828956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-124f5626-5847-421d-86e4-c7fa0b553335 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:12.830257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-124f5626-5847-421d-86e4-c7fa0b553335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:12.964089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 671/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:28:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:17.023461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=57 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:18.146890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874b013-f857-4305-ba50-fd9c197aa3d8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:18.148046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874b013-f857-4305-ba50-fd9c197aa3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:18.148549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874b013-f857-4305-ba50-fd9c197aa3d8 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:18.150556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 681/1352] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:28:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:18.156801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac812-a4c7-4610-9423-da8d42627026 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:18.157390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac812-a4c7-4610-9423-da8d42627026 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:18.157724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac812-a4c7-4610-9423-da8d42627026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:18.190191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fedac812-a4c7-4610-9423-da8d42627026 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:18.193471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fedac812-a4c7-4610-9423-da8d42627026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:18.209695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 672/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:19.546553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669652dc-b3fb-4acd-967a-c177ab0a0497 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:19.547049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669652dc-b3fb-4acd-967a-c177ab0a0497 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:19.547049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669652dc-b3fb-4acd-967a-c177ab0a0497 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:19.548368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 682/1354] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:19.554640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510642e-97c4-4042-bbfd-9c45c3041f8b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:19.555154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510642e-97c4-4042-bbfd-9c45c3041f8b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:19.555294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510642e-97c4-4042-bbfd-9c45c3041f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:19.574321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9510642e-97c4-4042-bbfd-9c45c3041f8b None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:19.575553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9510642e-97c4-4042-bbfd-9c45c3041f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:19.584605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 673/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:23.911397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc340-2d0f-421f-83f7-f4112a9a4cfe None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:23.912206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc340-2d0f-421f-83f7-f4112a9a4cfe None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:23.912206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc340-2d0f-421f-83f7-f4112a9a4cfe None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:23.913206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 683/1356] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:23.922254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef8ef0-97ab-4e68-b49c-1bfce509ee0c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:23.922768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef8ef0-97ab-4e68-b49c-1bfce509ee0c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:23.923150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef8ef0-97ab-4e68-b49c-1bfce509ee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:23.952860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-48ef8ef0-97ab-4e68-b49c-1bfce509ee0c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:23.954991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-48ef8ef0-97ab-4e68-b49c-1bfce509ee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:23.968683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 674/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:24.023969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69e6e-05b9-4540-833a-b0c0e91f5388 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:24.024381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69e6e-05b9-4540-833a-b0c0e91f5388 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:24.024934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c69e6e-05b9-4540-833a-b0c0e91f5388 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:24.027291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 684/1358] 10.222.0.48 () {54 vars in 961 bytes} [Fri May 27 09:28:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:24.044097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076a25-3eee-4193-a814-365f7d40d25f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:24.044601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076a25-3eee-4193-a814-365f7d40d25f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:24.045116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076a25-3eee-4193-a814-365f7d40d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:24.114860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-12076a25-3eee-4193-a814-365f7d40d25f None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:24.116539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-12076a25-3eee-4193-a814-365f7d40d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:24.208505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:24.270236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 675/1359] 10.222.0.48 () {58 vars in 1058 bytes} [Fri May 27 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:24.283750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c69e6e-05b9-4540-833a-b0c0e91f5388 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:24.301284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c69e6e-05b9-4540-833a-b0c0e91f5388 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:24.308802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:24.309634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:24.309915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:24.310183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:24.328429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:24.331167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418602d7-55a5-4520-9cc0-d00f7ad33734 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:24.344388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 685/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:28:27.262002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bd33-ccc5-492e-8b58-520d4e5d88ec None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:27.262650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bd33-ccc5-492e-8b58-520d4e5d88ec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:27.262650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bd33-ccc5-492e-8b58-520d4e5d88ec None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:27.266702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 676/1361] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:27.271156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920216ae-94c1-4416-86d2-617c4b352278 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:27.271914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920216ae-94c1-4416-86d2-617c4b352278 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:27.271914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920216ae-94c1-4416-86d2-617c4b352278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:27.291644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-920216ae-94c1-4416-86d2-617c4b352278 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:27.293251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-920216ae-94c1-4416-86d2-617c4b352278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:27.301426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 686/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:28:27.348296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36bd33-ccc5-492e-8b58-520d4e5d88ec None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:27.363584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36bd33-ccc5-492e-8b58-520d4e5d88ec None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:27.369111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:27.369753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:27.370037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:27.370353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:27.380909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:27.384871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e867-e104-4a60-b8cd-9eb74ce66bb9 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:27.393906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 677/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:30.390530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19161e-8f9a-4aa3-89f3-526b8205a443 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:30.391066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19161e-8f9a-4aa3-89f3-526b8205a443 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:30.391066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19161e-8f9a-4aa3-89f3-526b8205a443 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:30.392784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 687/1364] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:30.399470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb367bf-96f4-4674-a628-695d8ed30637 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:30.400029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb367bf-96f4-4674-a628-695d8ed30637 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:30.400029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb367bf-96f4-4674-a628-695d8ed30637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:30.420495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3bb367bf-96f4-4674-a628-695d8ed30637 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:30.421774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3bb367bf-96f4-4674-a628-695d8ed30637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:30.430141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 678/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:30.480663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c30f-92ca-40ad-9112-b38f08fbfa67 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:30.481044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c30f-92ca-40ad-9112-b38f08fbfa67 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:30.481382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60c30f-92ca-40ad-9112-b38f08fbfa67 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:30.482936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 688/1366] 10.222.0.48 () {54 vars in 961 bytes} [Fri May 27 09:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:30.488932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a308b4-f947-442d-b636-5b800515e024 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:30.489220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a308b4-f947-442d-b636-5b800515e024 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:30.489500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a308b4-f947-442d-b636-5b800515e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:30.509405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-08a308b4-f947-442d-b636-5b800515e024 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:30.510664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08a308b4-f947-442d-b636-5b800515e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:30.521000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 679/1367] 10.222.0.48 () {58 vars in 1058 bytes} [Fri May 27 09:28:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:30.529518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60c30f-92ca-40ad-9112-b38f08fbfa67 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:30.544646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60c30f-92ca-40ad-9112-b38f08fbfa67 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:30.550441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:30.551141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:30.551425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:30.551698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:30.561192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:30.562834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81323-ad9a-48e4-be13-6f476c2193a2 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:30.573954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 689/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:33.776217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331d18-f495-4283-9d0b-5147b788861a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:33.777555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331d18-f495-4283-9d0b-5147b788861a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:33.778052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331d18-f495-4283-9d0b-5147b788861a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:33.780657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 680/1369] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:33.790086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b311ec-fe07-44ad-b538-8b070f7f67a1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:33.790864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b311ec-fe07-44ad-b538-8b070f7f67a1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:33.791068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b311ec-fe07-44ad-b538-8b070f7f67a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:33.821603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e6b311ec-fe07-44ad-b538-8b070f7f67a1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:33.823784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e6b311ec-fe07-44ad-b538-8b070f7f67a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:33.838155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 690/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:33.914776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9331d18-f495-4283-9d0b-5147b788861a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:33.938873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9331d18-f495-4283-9d0b-5147b788861a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:33.948784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:33.949897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:33.950436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:33.950933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:33.967651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:33.970725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155e949-4414-4f36-a7e1-2c30dd2b94f8 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:33.989880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 681/1371] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:28:34.571776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de314185-1228-4b60-92ee-fa72329bf3bf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:34.572309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de314185-1228-4b60-92ee-fa72329bf3bf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:34.572859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de314185-1228-4b60-92ee-fa72329bf3bf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:34.575083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 691/1372] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:34.582787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d854-0450-4a6a-bdcb-b9f43d53d714 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:34.583294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d854-0450-4a6a-bdcb-b9f43d53d714 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:34.583824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d854-0450-4a6a-bdcb-b9f43d53d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:34.615856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1cb9d854-0450-4a6a-bdcb-b9f43d53d714 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:34.617677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1cb9d854-0450-4a6a-bdcb-b9f43d53d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:34.639905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 682/1373] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:28:35.131376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8098-912e-45e7-930e-98d48e07f4db None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:35.131376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8098-912e-45e7-930e-98d48e07f4db None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:35.131376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8098-912e-45e7-930e-98d48e07f4db None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:35.135680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 692/1374] 10.222.0.48 () {54 vars in 963 bytes} [Fri May 27 09:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:35.141294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd546-cbd9-4541-bde9-ea0f2d6c3c84 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:35.141294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd546-cbd9-4541-bde9-ea0f2d6c3c84 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:35.141294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bd546-cbd9-4541-bde9-ea0f2d6c3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:35.239405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-979bd546-cbd9-4541-bde9-ea0f2d6c3c84 None None] MFA Rules not processed for user `3963b8bacbc74ef2a579be684a219ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:35.240622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-979bd546-cbd9-4541-bde9-ea0f2d6c3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:35.312842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 683/1375] 10.222.0.48 () {58 vars in 1060 bytes} [Fri May 27 09:28:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:35.320694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b8098-912e-45e7-930e-98d48e07f4db None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:35.334515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b8098-912e-45e7-930e-98d48e07f4db None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:35.338190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:35.338881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:35.339165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:35.339531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:35.350415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:35.351963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7e8ea-2d2a-4a15-8e93-46e87e1ad16d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:35.362492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 693/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:36.702622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389084e-38ed-41cf-ab60-f39bf3fa26e4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:36.703410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389084e-38ed-41cf-ab60-f39bf3fa26e4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:36.703410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389084e-38ed-41cf-ab60-f39bf3fa26e4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:36.704544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 684/1377] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:36.711039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd20c4-8302-49fe-ac85-a65e7f035647 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:36.711614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd20c4-8302-49fe-ac85-a65e7f035647 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:36.711742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd20c4-8302-49fe-ac85-a65e7f035647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:36.730563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-20fd20c4-8302-49fe-ac85-a65e7f035647 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:36.731779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-20fd20c4-8302-49fe-ac85-a65e7f035647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:36.740427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 694/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:28:36.786640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389084e-38ed-41cf-ab60-f39bf3fa26e4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:36.801542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389084e-38ed-41cf-ab60-f39bf3fa26e4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:36.807480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:36.808125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:36.808407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:36.808675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:36.818784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:36.820404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f33edf-1576-4928-a642-48a2dba6e4ad service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:36.831211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 685/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:39.317439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f6b-5a73-41ac-9f1f-cb7943099ff9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:39.318003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f6b-5a73-41ac-9f1f-cb7943099ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:39.318003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f6b-5a73-41ac-9f1f-cb7943099ff9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:39.319384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 695/1380] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:39.324870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cbb1-b7aa-49df-a336-4590bf73e6c6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:39.325341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cbb1-b7aa-49df-a336-4590bf73e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:39.325439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9cbb1-b7aa-49df-a336-4590bf73e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:39.355301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-bab9cbb1-b7aa-49df-a336-4590bf73e6c6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:39.356550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bab9cbb1-b7aa-49df-a336-4590bf73e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:39.367135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 686/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:39.418868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c7f6b-5a73-41ac-9f1f-cb7943099ff9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:39.437718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c7f6b-5a73-41ac-9f1f-cb7943099ff9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:39.451186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:39.452227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:39.452737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:39.453256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:39.466874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:39.470514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db2bf8-7555-4ebf-acd2-1c6a8072efe3 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:39.484808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 696/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:42.512641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49780e9-4a57-4105-8c02-1e166040a542 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:42.513202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49780e9-4a57-4105-8c02-1e166040a542 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:42.513202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49780e9-4a57-4105-8c02-1e166040a542 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:42.514573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 687/1383] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:42.521811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5293ff-a08d-45fb-a9c5-4d81aaa2ee34 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:42.522362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5293ff-a08d-45fb-a9c5-4d81aaa2ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:42.522443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5293ff-a08d-45fb-a9c5-4d81aaa2ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:42.547221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1a5293ff-a08d-45fb-a9c5-4d81aaa2ee34 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:42.548462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1a5293ff-a08d-45fb-a9c5-4d81aaa2ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:42.557287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 697/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:42.601545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49780e9-4a57-4105-8c02-1e166040a542 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:42.621395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49780e9-4a57-4105-8c02-1e166040a542 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:42.627850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:42.628508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:42.628809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:42.629087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:42.639121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:42.640732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7554799-a264-4ed2-b83c-bc92491460e6 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:42.652649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 688/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:45.190149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba59-dea2-4dff-8803-1f4f0975093e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:45.190149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba59-dea2-4dff-8803-1f4f0975093e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:45.190149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba59-dea2-4dff-8803-1f4f0975093e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:45.190149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 698/1386] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:45.199469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6cf28-b53f-4737-836f-c3a175fef369 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:45.203640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6cf28-b53f-4737-836f-c3a175fef369 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:45.203640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6cf28-b53f-4737-836f-c3a175fef369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:45.230461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-52f6cf28-b53f-4737-836f-c3a175fef369 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:45.231779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-52f6cf28-b53f-4737-836f-c3a175fef369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:45.244434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 689/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:45.292858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ba59-dea2-4dff-8803-1f4f0975093e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:45.317540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ba59-dea2-4dff-8803-1f4f0975093e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:45.325842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:45.327383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:45.328124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:45.329145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:45.340072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:45.341564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b88f4-5224-4e31-9786-cd89a979e5db service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:45.352850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 699/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:48.388434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759813-bb37-47cc-a134-09b027e13377 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:48.389111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759813-bb37-47cc-a134-09b027e13377 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:48.389111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759813-bb37-47cc-a134-09b027e13377 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:48.390409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 690/1389] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:28:48.396316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbfecb-1c07-4627-88fb-4b064d1cecd6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:48.396837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbfecb-1c07-4627-88fb-4b064d1cecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:48.396837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbfecb-1c07-4627-88fb-4b064d1cecd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:48.403564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=43 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:48.416973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7fbbfecb-1c07-4627-88fb-4b064d1cecd6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:48.418179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7fbbfecb-1c07-4627-88fb-4b064d1cecd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:48.426512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 700/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:28:48.473527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759813-bb37-47cc-a134-09b027e13377 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:48.490680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759813-bb37-47cc-a134-09b027e13377 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:48.496718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:48.497264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:48.497548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:48.497824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:48.508100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:48.511189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976580b-a24c-44c1-933c-404791298fb0 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:48.521311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 691/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:28:51.231917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f714-024a-4cd1-81a5-0c560ccc8a30 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:51.240192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f714-024a-4cd1-81a5-0c560ccc8a30 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:51.240192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f714-024a-4cd1-81a5-0c560ccc8a30 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:51.240192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 701/1392] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:28:51.240774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bd8f3-6a93-4bd4-ae71-0ac13579bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:51.241503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bd8f3-6a93-4bd4-ae71-0ac13579bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:51.241686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bd8f3-6a93-4bd4-ae71-0ac13579bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:51.260807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-536bd8f3-6a93-4bd4-ae71-0ac13579bcb1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:51.262017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-536bd8f3-6a93-4bd4-ae71-0ac13579bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:51.270227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 692/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:51.317473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f714-024a-4cd1-81a5-0c560ccc8a30 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:51.334261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f714-024a-4cd1-81a5-0c560ccc8a30 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:51.340854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:51.341550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:51.341870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:51.342180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:51.352172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:51.353749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a26b57-3757-4ad5-bee2-d0c4a44d31fb service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:51.364820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 702/1394] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:52.510660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c26e6-6587-41a0-98df-4f4aa5f0300f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:52.511242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c26e6-6587-41a0-98df-4f4aa5f0300f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:52.511242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c26e6-6587-41a0-98df-4f4aa5f0300f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:52.512748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 693/1395] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:52.518420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b018f-fb78-4e02-84c3-6f6e879610c1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:52.518924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b018f-fb78-4e02-84c3-6f6e879610c1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:52.519582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b018f-fb78-4e02-84c3-6f6e879610c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:52.538592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-df8b018f-fb78-4e02-84c3-6f6e879610c1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:52.539843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-df8b018f-fb78-4e02-84c3-6f6e879610c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:52.554315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 703/1396] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:52.615473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe1ee6-2164-433d-8759-9cbecd15c274 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:52.615776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe1ee6-2164-433d-8759-9cbecd15c274 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:52.616121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe1ee6-2164-433d-8759-9cbecd15c274 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:52.617586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 694/1397] 10.222.0.48 () {54 vars in 962 bytes} [Fri May 27 09:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:52.623046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d07e-f698-4848-b80b-74ea722ea1ba None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:52.623336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d07e-f698-4848-b80b-74ea722ea1ba None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:52.623613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d07e-f698-4848-b80b-74ea722ea1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:52.676559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-49b8d07e-f698-4848-b80b-74ea722ea1ba None None] MFA Rules not processed for user `ab8e09e92e284747b80ca816b6e15a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:52.677780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-49b8d07e-f698-4848-b80b-74ea722ea1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:52.747159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 704/1398] 10.222.0.48 () {58 vars in 1059 bytes} [Fri May 27 09:28:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:52.756415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe1ee6-2164-433d-8759-9cbecd15c274 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:52.767953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe1ee6-2164-433d-8759-9cbecd15c274 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:52.772099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:52.772743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:52.773021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:52.773293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:52.788148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:52.789646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78d9ec-f1ef-498d-99a1-91d8af8b4de1 service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:52.800900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 695/1399] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:28:53.557453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651074-169b-448c-bb80-8aa8cdee4fdf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.558254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651074-169b-448c-bb80-8aa8cdee4fdf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.558254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651074-169b-448c-bb80-8aa8cdee4fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.621552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c6651074-169b-448c-bb80-8aa8cdee4fdf None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:53.625507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c6651074-169b-448c-bb80-8aa8cdee4fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:53.718514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 705/1400] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:53.768191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc593-7213-48f1-bc60-35240d5516a8 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.768728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc593-7213-48f1-bc60-35240d5516a8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.770394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc593-7213-48f1-bc60-35240d5516a8 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.771256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 696/1401] 10.222.0.48 () {54 vars in 942 bytes} [Fri May 27 09:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:28:53.778825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.779202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.779566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.822315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:53.840126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] MFA Rules not processed for user `444436f77ef74d9fb825aafa19ccc146`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:53.841445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:53.865730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce357bd-9f41-4de0-b492-cebffef8e736 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.866236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce357bd-9f41-4de0-b492-cebffef8e736 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.866593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce357bd-9f41-4de0-b492-cebffef8e736 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.868062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 697/1402] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:28:53.875593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.876873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.877663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.897309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:28:53.904686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:53.906582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:53.919245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 698/1403] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:53.934376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 706/1404] 10.222.0.48 () {58 vars in 1039 bytes} [Fri May 27 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:53.943480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:53.960031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffb2b5-e6c8-4abf-ad0c-e035a07aeb81 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:53.966264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:53.966763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:53.967150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:53.967339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:53.977324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:53.978983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2aa9-4d8a-458b-80f6-fc0a9cb3c430 service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:53.984091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:53.988730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 699/1405] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:28:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:28:54.001161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493dfdf-d0c5-4b70-990b-3c3245b1a0c2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:54.008518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:54.009286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:54.010246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:54.010246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:54.023471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:54.029995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a424a2-f8a8-44a8-a154-7564ea98d22e service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:54.040320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 707/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:28:57.120367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a3b4-8d90-4f66-ad58-6dfbf5ca2665 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:57.120367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a3b4-8d90-4f66-ad58-6dfbf5ca2665 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:57.121118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a3b4-8d90-4f66-ad58-6dfbf5ca2665 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:57.126253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 700/1407] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:28:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:28:57.129721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87fd1-d6e4-4c00-9c5f-2857e3f31c28 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:57.130699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87fd1-d6e4-4c00-9c5f-2857e3f31c28 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:57.130699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c87fd1-d6e4-4c00-9c5f-2857e3f31c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:57.153460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-84c87fd1-d6e4-4c00-9c5f-2857e3f31c28 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:28:57.154798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-84c87fd1-d6e4-4c00-9c5f-2857e3f31c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:28:57.172658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 708/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:28:57.219396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452a3b4-8d90-4f66-ad58-6dfbf5ca2665 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:28:57.240855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452a3b4-8d90-4f66-ad58-6dfbf5ca2665 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:28:57.247256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:28:57.248961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:28:57.249272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:28:57.249547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:28:57.262365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:28:57.263806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b061b3-363b-42ff-ac50-c6bd45e91588 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:28:57.279232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 701/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:29:07.147537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=42 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:29:07.273560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=12 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:29:19.177709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f77b4-5fee-4040-b17b-c5efba6ef6b0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:19.178279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f77b4-5fee-4040-b17b-c5efba6ef6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:19.178279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f77b4-5fee-4040-b17b-c5efba6ef6b0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:19.179924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 709/1410] 10.222.0.48 () {54 vars in 918 bytes} [Fri May 27 09:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:19.185647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074819a-9eb7-4103-819a-01994310e289 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:19.186114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074819a-9eb7-4103-819a-01994310e289 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:19.186375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074819a-9eb7-4103-819a-01994310e289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:19.204511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f074819a-9eb7-4103-819a-01994310e289 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:19.205715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f074819a-9eb7-4103-819a-01994310e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:19.214893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 702/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Fri May 27 09:29:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:19.229810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76477ae9-bab8-4ac0-8c44-bfcfb733df35 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:19.230104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76477ae9-bab8-4ac0-8c44-bfcfb733df35 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:19.230417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76477ae9-bab8-4ac0-8c44-bfcfb733df35 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:19.231776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 710/1412] 10.222.0.48 () {54 vars in 918 bytes} [Fri May 27 09:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:19.236474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a0aa-60e3-4250-a889-915436deae24 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:19.236757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a0aa-60e3-4250-a889-915436deae24 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:19.237026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a0aa-60e3-4250-a889-915436deae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:19.255880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4970a0aa-60e3-4250-a889-915436deae24 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:19.257064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4970a0aa-60e3-4250-a889-915436deae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:19.265969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 703/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Fri May 27 09:29:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:22.064055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e821c-6291-40a8-84cb-6341528b1e50 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:22.064634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e821c-6291-40a8-84cb-6341528b1e50 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:22.064634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e821c-6291-40a8-84cb-6341528b1e50 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:22.066664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:22.072006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3e3c-7f96-4772-928e-1db81bef446c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:22.072482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3e3c-7f96-4772-928e-1db81bef446c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:22.072579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3e3c-7f96-4772-928e-1db81bef446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:22.089954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-049b3e3c-7f96-4772-928e-1db81bef446c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:22.091190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-049b3e3c-7f96-4772-928e-1db81bef446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:22.099594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:22.148637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e821c-6291-40a8-84cb-6341528b1e50 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:22.163988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e821c-6291-40a8-84cb-6341528b1e50 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:22.169959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:22.170640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:22.170920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:22.171193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:22.181706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:22.183390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac6621-a2b4-4c9b-b3fd-5ce08ba06db6 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:22.193622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 712/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:23.180258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435df7-8dfe-40cf-bc07-f855bcbde089 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:23.180822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435df7-8dfe-40cf-bc07-f855bcbde089 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:23.180822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435df7-8dfe-40cf-bc07-f855bcbde089 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:23.182165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:23.187466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471225-ea1e-499e-8718-6976b8a566fc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:23.187900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471225-ea1e-499e-8718-6976b8a566fc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:23.188022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6471225-ea1e-499e-8718-6976b8a566fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:23.205780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f6471225-ea1e-499e-8718-6976b8a566fc None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:23.207016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f6471225-ea1e-499e-8718-6976b8a566fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:23.214780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:23.375158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435df7-8dfe-40cf-bc07-f855bcbde089 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:23.389615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435df7-8dfe-40cf-bc07-f855bcbde089 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:23.394874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:23.395492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:23.395771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:23.396042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:23.403987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:23.405670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344c782-2eef-4efe-9c53-68a8e8eb4974 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:23.415161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 706/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:24.357494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d1a1-6a5d-4366-b49f-55bbc2affa92 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:24.358031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d1a1-6a5d-4366-b49f-55bbc2affa92 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:24.358031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2d1a1-6a5d-4366-b49f-55bbc2affa92 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:24.359368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:24.364947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2dc0e-8dca-493c-8f25-3f5efef159de None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:24.365436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2dc0e-8dca-493c-8f25-3f5efef159de None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:24.365536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2dc0e-8dca-493c-8f25-3f5efef159de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:24.383407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-eeb2dc0e-8dca-493c-8f25-3f5efef159de None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:24.387286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-eeb2dc0e-8dca-493c-8f25-3f5efef159de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:24.392215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:24.544583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2d1a1-6a5d-4366-b49f-55bbc2affa92 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:24.560204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2d1a1-6a5d-4366-b49f-55bbc2affa92 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:24.566699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:24.567349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:24.567626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:24.567917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:24.577545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:24.579157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc6dfe-6ffd-4196-909a-d9b8f7152a12 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:24.589552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 715/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:25.517673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80ae7-ff3d-490d-b184-8870fb62751f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:25.518230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80ae7-ff3d-490d-b184-8870fb62751f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:25.518230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80ae7-ff3d-490d-b184-8870fb62751f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:25.520193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:25.526019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0637-92ef-40d9-b95b-96174b204eda None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:25.526614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0637-92ef-40d9-b95b-96174b204eda None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:25.526614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0637-92ef-40d9-b95b-96174b204eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:25.546055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-88ec0637-92ef-40d9-b95b-96174b204eda None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:25.547315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-88ec0637-92ef-40d9-b95b-96174b204eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:25.556108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:25.709621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80ae7-ff3d-490d-b184-8870fb62751f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:25.724730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80ae7-ff3d-490d-b184-8870fb62751f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:25.730499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:25.731141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:25.731424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:25.731705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:25.740985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:25.742761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf7b81-8b6b-4a69-afda-79ef9e28251c service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:25.752716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 709/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:26.670933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3611b4-aa95-4cb0-9148-179c1c438229 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:26.671555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3611b4-aa95-4cb0-9148-179c1c438229 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:26.671555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3611b4-aa95-4cb0-9148-179c1c438229 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:26.672855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:26.678188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0ce0-1bfb-41be-b0f8-6267ba93e66d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:26.678669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0ce0-1bfb-41be-b0f8-6267ba93e66d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:26.678772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0ce0-1bfb-41be-b0f8-6267ba93e66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:26.697776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0f2d0ce0-1bfb-41be-b0f8-6267ba93e66d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:26.699032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0f2d0ce0-1bfb-41be-b0f8-6267ba93e66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:26.707526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:26.857492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3611b4-aa95-4cb0-9148-179c1c438229 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:26.872381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3611b4-aa95-4cb0-9148-179c1c438229 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:26.877787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:26.878442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:26.878729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:26.879005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:26.887101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:26.888651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cd258-4897-4d34-a93e-8294bb9b9ccc service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:26.898176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 718/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:27.828154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c202ea-f014-40eb-916f-b41fc9844e0a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:27.828754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c202ea-f014-40eb-916f-b41fc9844e0a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:27.828754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c202ea-f014-40eb-916f-b41fc9844e0a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:27.829977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:27.835887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b3e6-81e9-47ea-a551-b3cf33afa480 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:27.836408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b3e6-81e9-47ea-a551-b3cf33afa480 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:27.836578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b3e6-81e9-47ea-a551-b3cf33afa480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:27.856219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2e05b3e6-81e9-47ea-a551-b3cf33afa480 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:27.857474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2e05b3e6-81e9-47ea-a551-b3cf33afa480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:27.865903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:28.019107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c202ea-f014-40eb-916f-b41fc9844e0a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:28.033311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c202ea-f014-40eb-916f-b41fc9844e0a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:28.038797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:28.039425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:28.039702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:28.039975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:28.048886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:28.051283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9a125-498a-4968-8e03-e793f0bdab3b service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:28.062112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 712/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:28.986120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa673-da13-40a9-8dbf-f0e4558954aa None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:28.986691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa673-da13-40a9-8dbf-f0e4558954aa None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:28.986691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa673-da13-40a9-8dbf-f0e4558954aa None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:28.988012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:28.993053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43772b-c286-4fbd-98aa-a179b12ce6f4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:28.993535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43772b-c286-4fbd-98aa-a179b12ce6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:28.993633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43772b-c286-4fbd-98aa-a179b12ce6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:29.012092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-de43772b-c286-4fbd-98aa-a179b12ce6f4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:29.013292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-de43772b-c286-4fbd-98aa-a179b12ce6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:29.022130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:29.173405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa673-da13-40a9-8dbf-f0e4558954aa None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:29.189194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa673-da13-40a9-8dbf-f0e4558954aa None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:29.195313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:29.195952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:29.196231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:29.196506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:29.205341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:29.207592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf88a-3de8-4adc-9bca-53c92dfb629d service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:29.217046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 721/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:30.158416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678700-d34b-4fb2-b15b-1928bce2647e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:30.158974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678700-d34b-4fb2-b15b-1928bce2647e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:30.158974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678700-d34b-4fb2-b15b-1928bce2647e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:30.160977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:30.166876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ae6ba-e75e-4f53-a516-63bdcd8e6e5a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:30.167400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ae6ba-e75e-4f53-a516-63bdcd8e6e5a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:30.167400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ae6ba-e75e-4f53-a516-63bdcd8e6e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:30.185901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-890ae6ba-e75e-4f53-a516-63bdcd8e6e5a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:30.187137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-890ae6ba-e75e-4f53-a516-63bdcd8e6e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:30.195305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:30.345063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d678700-d34b-4fb2-b15b-1928bce2647e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:30.359985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d678700-d34b-4fb2-b15b-1928bce2647e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:30.365638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:30.366339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:30.366662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:30.366945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:30.375907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:30.377524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b382a4c-9cbc-4477-a49a-f5575d5b9977 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:30.388213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 715/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:31.357319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7e88-c4df-407c-9b78-7d46a65285f7 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:31.357900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7e88-c4df-407c-9b78-7d46a65285f7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:31.357900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf7e88-c4df-407c-9b78-7d46a65285f7 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:31.359294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:31.365108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cfbdb-0e24-4761-b6e8-b3d1e8b0251d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:31.365637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cfbdb-0e24-4761-b6e8-b3d1e8b0251d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:31.365782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cfbdb-0e24-4761-b6e8-b3d1e8b0251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:31.384114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2f3cfbdb-0e24-4761-b6e8-b3d1e8b0251d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:31.385309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2f3cfbdb-0e24-4761-b6e8-b3d1e8b0251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:31.393182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:31.544970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7e88-c4df-407c-9b78-7d46a65285f7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:31.559077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf7e88-c4df-407c-9b78-7d46a65285f7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:31.564649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:31.565311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:31.565597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:31.565867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:31.574576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:31.576155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc136f-4036-4cfc-bf5f-f7deb8dfd16c service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:31.586134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 724/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:33.657158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e51bf-a01a-4bb0-a3b2-cd6f2e9b7516 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:33.657718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e51bf-a01a-4bb0-a3b2-cd6f2e9b7516 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:33.657718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e51bf-a01a-4bb0-a3b2-cd6f2e9b7516 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:33.659628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:33.665114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669195-923c-4f72-bfa3-3cc77cfc89c3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:33.665607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669195-923c-4f72-bfa3-3cc77cfc89c3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:33.665731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669195-923c-4f72-bfa3-3cc77cfc89c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:33.684912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ed669195-923c-4f72-bfa3-3cc77cfc89c3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:33.686458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ed669195-923c-4f72-bfa3-3cc77cfc89c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:33.694794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:33.733040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e51bf-a01a-4bb0-a3b2-cd6f2e9b7516 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:33.744622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e51bf-a01a-4bb0-a3b2-cd6f2e9b7516 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:33.748372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:33.749017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:33.749298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:33.749573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:33.760184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:33.761679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65183c-4457-42a6-85f7-4776f3ddce5f service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:33.773112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 718/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:36.162126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831355-d716-4935-ae4c-328b7c88dad0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:36.162744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831355-d716-4935-ae4c-328b7c88dad0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:36.162744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831355-d716-4935-ae4c-328b7c88dad0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:36.164112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:36.169633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46e3c-b508-4702-8453-60e13a34b20d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:36.170073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46e3c-b508-4702-8453-60e13a34b20d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:36.170180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46e3c-b508-4702-8453-60e13a34b20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:36.188457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-97e46e3c-b508-4702-8453-60e13a34b20d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:36.190023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-97e46e3c-b508-4702-8453-60e13a34b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:36.198691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:36.236081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831355-d716-4935-ae4c-328b7c88dad0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:36.248112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831355-d716-4935-ae4c-328b7c88dad0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:36.251885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:36.252520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:36.253396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:36.253668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:36.263255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:36.264697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd352e-aa85-4ab1-b8e9-5464a04866d6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:36.275344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 727/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:38.877675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acb88-a83e-4f20-b7b3-87afbde9f149 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:38.878296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acb88-a83e-4f20-b7b3-87afbde9f149 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:38.878296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acb88-a83e-4f20-b7b3-87afbde9f149 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:38.879601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:38.884947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdbc0-fcb2-4570-8ef4-e3c80ea51202 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:38.885378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdbc0-fcb2-4570-8ef4-e3c80ea51202 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:38.885481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdbc0-fcb2-4570-8ef4-e3c80ea51202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:38.903051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-27bfdbc0-fcb2-4570-8ef4-e3c80ea51202 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:38.904204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-27bfdbc0-fcb2-4570-8ef4-e3c80ea51202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:38.912449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:38.951492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3acb88-a83e-4f20-b7b3-87afbde9f149 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:38.961976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3acb88-a83e-4f20-b7b3-87afbde9f149 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:38.965425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:38.966052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:38.966368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:38.966676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:38.976525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:38.977996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72540516-8b5d-4e1e-8747-5f8224a29cbc service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:38.988053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 721/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:41.407059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29340-66d4-4951-a608-162ce53ddc0b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:41.407637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29340-66d4-4951-a608-162ce53ddc0b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:41.407637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29340-66d4-4951-a608-162ce53ddc0b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:41.409009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:41.414349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb774cf-75c6-40e9-a918-90759a6d4948 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:41.414821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb774cf-75c6-40e9-a918-90759a6d4948 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:41.414930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb774cf-75c6-40e9-a918-90759a6d4948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:41.433573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0fb774cf-75c6-40e9-a918-90759a6d4948 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:41.434793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0fb774cf-75c6-40e9-a918-90759a6d4948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:41.443700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:41.481667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff29340-66d4-4951-a608-162ce53ddc0b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:41.492743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff29340-66d4-4951-a608-162ce53ddc0b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:41.496256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:41.496876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:41.497156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:41.497433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:41.506862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:41.508288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebbac7-86f5-4040-8ec7-bd1235dfffa2 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:41.517607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 730/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:43.959465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eff80-80f4-4c12-80ad-c3aa30d66790 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:43.960011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eff80-80f4-4c12-80ad-c3aa30d66790 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:43.960011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eff80-80f4-4c12-80ad-c3aa30d66790 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:43.961302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:43.966754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8dcbf-0f36-422f-9654-c2d24dc8dcf9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:43.967235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8dcbf-0f36-422f-9654-c2d24dc8dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:43.967358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8dcbf-0f36-422f-9654-c2d24dc8dcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:43.986305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c9e8dcbf-0f36-422f-9654-c2d24dc8dcf9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:43.988102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c9e8dcbf-0f36-422f-9654-c2d24dc8dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:43.995933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:44.032727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197eff80-80f4-4c12-80ad-c3aa30d66790 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:44.044252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197eff80-80f4-4c12-80ad-c3aa30d66790 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:44.047694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:44.048302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:44.048576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:44.048843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:44.058468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:44.059889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6397992-25b0-429e-86b0-ee945112d858 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:44.069525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 724/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:46.356696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d380a7-8613-47db-bb72-8eb5ab6beb43 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:46.357266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d380a7-8613-47db-bb72-8eb5ab6beb43 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:46.357266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d380a7-8613-47db-bb72-8eb5ab6beb43 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:46.358658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:46.364636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389abec8-fb77-4b77-8c53-4fae61f2e713 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:46.365098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389abec8-fb77-4b77-8c53-4fae61f2e713 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:46.365219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389abec8-fb77-4b77-8c53-4fae61f2e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:46.382940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-389abec8-fb77-4b77-8c53-4fae61f2e713 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:46.384131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-389abec8-fb77-4b77-8c53-4fae61f2e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:46.391826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:46.429652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d380a7-8613-47db-bb72-8eb5ab6beb43 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:46.441367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d380a7-8613-47db-bb72-8eb5ab6beb43 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:46.445127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:46.445763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:46.446041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:46.446375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:46.456399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:46.457816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e125d7-002f-48ee-a7e9-f4f4416ce5b9 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:46.467425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 733/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:48.745368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bf256-0bf1-4be4-85af-717e9aec26c4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:48.745946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bf256-0bf1-4be4-85af-717e9aec26c4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:48.746050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bf256-0bf1-4be4-85af-717e9aec26c4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:48.747440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:48.753447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757ff5-9b5d-414b-b819-ce1fdf8c93e2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:48.753927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757ff5-9b5d-414b-b819-ce1fdf8c93e2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:48.754069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de757ff5-9b5d-414b-b819-ce1fdf8c93e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:48.760036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=70 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:29:48.772763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-de757ff5-9b5d-414b-b819-ce1fdf8c93e2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:48.773976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-de757ff5-9b5d-414b-b819-ce1fdf8c93e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:48.781872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:48.819782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bf256-0bf1-4be4-85af-717e9aec26c4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:48.825162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=73 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:29:48.831556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bf256-0bf1-4be4-85af-717e9aec26c4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:48.835502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:48.836169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:48.836449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:48.836719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:48.846141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:48.848220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63f25-2329-4608-8c98-8c4ac98e355f service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:48.857874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 727/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:51.115166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031c47-def1-41fe-b265-cd296bbcce6d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:51.115757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031c47-def1-41fe-b265-cd296bbcce6d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:51.115757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031c47-def1-41fe-b265-cd296bbcce6d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:51.117097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:51.122706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99006d4b-b2a8-4a1d-b542-7987d8a2ac21 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:51.123163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99006d4b-b2a8-4a1d-b542-7987d8a2ac21 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:51.123298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99006d4b-b2a8-4a1d-b542-7987d8a2ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:51.140908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-99006d4b-b2a8-4a1d-b542-7987d8a2ac21 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:51.142521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-99006d4b-b2a8-4a1d-b542-7987d8a2ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:51.151587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:51.188644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031c47-def1-41fe-b265-cd296bbcce6d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:51.199328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031c47-def1-41fe-b265-cd296bbcce6d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:51.202859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:51.203482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:51.203770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:51.204028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:51.213599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:51.215065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09bb1a-7397-4b19-abc0-1fd2f774f059 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:51.225296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 736/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:53.451503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0750-0223-4961-bfed-cc65929bdfd5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:53.452083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0750-0223-4961-bfed-cc65929bdfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:53.452083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a0750-0223-4961-bfed-cc65929bdfd5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:53.453475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:53.459332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae429-ec2e-440b-ba64-79c9ad07dc7f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:53.459795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae429-ec2e-440b-ba64-79c9ad07dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:53.459892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae429-ec2e-440b-ba64-79c9ad07dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:53.477860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-207ae429-ec2e-440b-ba64-79c9ad07dc7f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:53.479081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-207ae429-ec2e-440b-ba64-79c9ad07dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:53.486814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:53.522542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a0750-0223-4961-bfed-cc65929bdfd5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:53.533575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a0750-0223-4961-bfed-cc65929bdfd5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:53.537734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:53.538412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:53.538695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:53.538967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:53.548381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:53.550393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814347e-b70e-48c9-883e-56d46fad01dc service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:53.559782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 730/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:29:55.808314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52fbf5-7734-4876-a64c-0b8ab083107a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:55.808881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52fbf5-7734-4876-a64c-0b8ab083107a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:55.808881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52fbf5-7734-4876-a64c-0b8ab083107a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:55.810528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:29:55.816498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bfd34-a7c7-4947-ac60-68c3bea057e0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:55.816923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bfd34-a7c7-4947-ac60-68c3bea057e0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:55.817023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bfd34-a7c7-4947-ac60-68c3bea057e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:55.835151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-097bfd34-a7c7-4947-ac60-68c3bea057e0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:29:55.836279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-097bfd34-a7c7-4947-ac60-68c3bea057e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:29:55.844438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:29:55.881299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52fbf5-7734-4876-a64c-0b8ab083107a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:29:55.891994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52fbf5-7734-4876-a64c-0b8ab083107a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:29:55.896287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:29:55.896910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:29:55.897189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:29:55.897461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:29:55.906670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:29:55.908048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997b7b-b066-46c2-8195-01d4e4ed5c13 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:29:55.917405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 739/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:05.829601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=17 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:30:05.913273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=17 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:30:29.300622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d331-be23-4148-be14-0972e2913d5d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:29.301229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d331-be23-4148-be14-0972e2913d5d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:29.301229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8d331-be23-4148-be14-0972e2913d5d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:29.302580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 732/1471] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:30:29.308203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dae782-5c60-4b32-a487-1a9d6ba83c86 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:29.308710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dae782-5c60-4b32-a487-1a9d6ba83c86 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:29.308814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dae782-5c60-4b32-a487-1a9d6ba83c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:29.327127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d9dae782-5c60-4b32-a487-1a9d6ba83c86 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:29.328344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d9dae782-5c60-4b32-a487-1a9d6ba83c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:29.337883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 740/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:29.385848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c88ca-870e-4b4c-8102-3692c7334479 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:29.386188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c88ca-870e-4b4c-8102-3692c7334479 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:29.386492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c88ca-870e-4b4c-8102-3692c7334479 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:29.387961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 733/1473] 10.222.0.48 () {54 vars in 962 bytes} [Fri May 27 09:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:29.393517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61dbf2-a13b-406f-8cea-2312470b1123 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:29.398834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61dbf2-a13b-406f-8cea-2312470b1123 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:29.398834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61dbf2-a13b-406f-8cea-2312470b1123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:29.413097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5f61dbf2-a13b-406f-8cea-2312470b1123 None None] MFA Rules not processed for user `ab8e09e92e284747b80ca816b6e15a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:29.414353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5f61dbf2-a13b-406f-8cea-2312470b1123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:29.420967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 741/1474] 10.222.0.48 () {58 vars in 1059 bytes} [Fri May 27 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:29.428997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c88ca-870e-4b4c-8102-3692c7334479 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:29.441132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c88ca-870e-4b4c-8102-3692c7334479 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:29.446696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:29.447347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:29.447629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:29.447907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:29.459505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:29.460953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3cd1e-7ba5-4424-a577-8e41e0ee7bef service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:29.471102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 734/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:30:31.849472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425ed9-784e-4e3e-ae11-cebc1623d856 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:31.850027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425ed9-784e-4e3e-ae11-cebc1623d856 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:31.850027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425ed9-784e-4e3e-ae11-cebc1623d856 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:31.851363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 742/1476] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:31.857215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb10-ebd5-46f7-bf0e-33fdce400b74 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:31.857752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb10-ebd5-46f7-bf0e-33fdce400b74 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:31.857913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6acb10-ebd5-46f7-bf0e-33fdce400b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:31.878816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fe6acb10-ebd5-46f7-bf0e-33fdce400b74 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:31.880025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fe6acb10-ebd5-46f7-bf0e-33fdce400b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:31.889181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 735/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:31.913496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425ed9-784e-4e3e-ae11-cebc1623d856 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:31.927521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425ed9-784e-4e3e-ae11-cebc1623d856 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:31.932823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:31.933571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:31.933854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:31.934128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:31.935328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:31.937134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:30:31.937134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:30:31.937134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:30:31.938446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c124d77-218c-415e-a30f-05dcadf10fb2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:31.963974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 743/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Fri May 27 09:30:31 2022] GET /identity/v3/projects => generated 2553 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:30:34.275474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52fbc-0773-4c20-a1cc-8f1d73e1967b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:34.276249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52fbc-0773-4c20-a1cc-8f1d73e1967b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:34.276249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52fbc-0773-4c20-a1cc-8f1d73e1967b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:34.277801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 736/1479] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:34.284887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a801c8-20f6-43bd-9f83-f05f08e39ec1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:34.285376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a801c8-20f6-43bd-9f83-f05f08e39ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:34.285489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a801c8-20f6-43bd-9f83-f05f08e39ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:34.305261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-65a801c8-20f6-43bd-9f83-f05f08e39ec1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:34.306948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-65a801c8-20f6-43bd-9f83-f05f08e39ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:34.317681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 744/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:34.369128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f52fbc-0773-4c20-a1cc-8f1d73e1967b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:34.384977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f52fbc-0773-4c20-a1cc-8f1d73e1967b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:34.392085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:34.392895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:34.393282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:34.393391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:34.403108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:34.404953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457411e-2954-47ad-a12f-97540a5b91be service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:34.416853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 737/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:36.904499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36501f-3dd4-4fed-aef3-39575ce53cfc None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:36.905067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36501f-3dd4-4fed-aef3-39575ce53cfc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:36.905067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36501f-3dd4-4fed-aef3-39575ce53cfc None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:36.906983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 745/1482] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:30:36.912810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e387c-0a8d-4dd7-8306-d7cc08c4db86 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:36.913309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e387c-0a8d-4dd7-8306-d7cc08c4db86 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:36.913426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e387c-0a8d-4dd7-8306-d7cc08c4db86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:36.932731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e22e387c-0a8d-4dd7-8306-d7cc08c4db86 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:36.934061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e22e387c-0a8d-4dd7-8306-d7cc08c4db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:36.943324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 738/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:36.979295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36501f-3dd4-4fed-aef3-39575ce53cfc None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:36.991816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36501f-3dd4-4fed-aef3-39575ce53cfc None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:36.995807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:36.996519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:36.996814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:36.997083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:37.008716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:37.010297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe530177-a451-494d-bed7-dd88ad212f99 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:37.020449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 746/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:39.324663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c929-9371-44e4-9db0-549b149bf3de None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:39.325313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c929-9371-44e4-9db0-549b149bf3de None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:39.325313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c929-9371-44e4-9db0-549b149bf3de None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:39.326629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 739/1485] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:39.333062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4397cb-ba47-4520-b80f-c837931715ad None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:39.333517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4397cb-ba47-4520-b80f-c837931715ad None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:39.333625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4397cb-ba47-4520-b80f-c837931715ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:39.353671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-da4397cb-ba47-4520-b80f-c837931715ad None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:39.354894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-da4397cb-ba47-4520-b80f-c837931715ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:39.363117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 747/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:39.403217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3c929-9371-44e4-9db0-549b149bf3de None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:39.414777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3c929-9371-44e4-9db0-549b149bf3de None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:39.418405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:39.419035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:39.419307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:39.419578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:39.428636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:39.430051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f848435-2632-4fd8-971a-fe5822ad75df service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:39.440568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 740/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:41.764363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915f0f-6ff7-4b03-8e35-23b94a8334c9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:41.764971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915f0f-6ff7-4b03-8e35-23b94a8334c9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:41.764971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9915f0f-6ff7-4b03-8e35-23b94a8334c9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:41.766309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 748/1488] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:41.771798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f4db-9b6b-4ef6-8c64-d570b93a0278 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:41.772259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f4db-9b6b-4ef6-8c64-d570b93a0278 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:41.772364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f4db-9b6b-4ef6-8c64-d570b93a0278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:41.790257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a0b8f4db-9b6b-4ef6-8c64-d570b93a0278 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:41.791443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a0b8f4db-9b6b-4ef6-8c64-d570b93a0278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:41.799855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 741/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:41.839261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9915f0f-6ff7-4b03-8e35-23b94a8334c9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:41.850791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9915f0f-6ff7-4b03-8e35-23b94a8334c9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:41.854266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:41.854889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:41.855165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:41.855438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:41.865164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:41.866613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783739e4-1f23-45cd-9820-6b778e246de4 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:41.875819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 749/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:44.514876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b3d-c321-4829-8709-2dbb7484c3c5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:44.515427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b3d-c321-4829-8709-2dbb7484c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:44.515427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b3d-c321-4829-8709-2dbb7484c3c5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:44.516776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 742/1491] 10.222.0.48 () {54 vars in 926 bytes} [Fri May 27 09:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:30:44.524363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d381-9282-416a-bba3-6031823f1db6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:44.524825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d381-9282-416a-bba3-6031823f1db6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:44.524928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d381-9282-416a-bba3-6031823f1db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:44.543400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2a07d381-9282-416a-bba3-6031823f1db6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:30:44.544588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2a07d381-9282-416a-bba3-6031823f1db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:30:44.552321 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 750/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Fri May 27 09:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:30:44.591007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6574b3d-c321-4829-8709-2dbb7484c3c5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:30:44.602359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6574b3d-c321-4829-8709-2dbb7484c3c5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:30:44.605837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:30:44.606503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:30:44.606787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:30:44.607062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:30:44.616078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:30:44.617510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f328e-5d06-4a38-8a7d-1429317640ac service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:30:44.627895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 743/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:30:54.538023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=33 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:30:54.622939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=33 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:32:36.440387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.440993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.440993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.459280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:32:36.460460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:36.468221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 751/1494] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:32:36.477635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.478291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.478291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.495835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:32:36.497019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:36.505587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 744/1495] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:32:36.510278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.520456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10447c11-8bec-4899-a3b1-69c2255876db None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.525132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.525788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.526064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.526428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.527140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:32:36.527442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.528444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.528444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:32:36.528444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:32:36.529312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:36.541157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 752/1496] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:32:36.549219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.558462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81062fa-a75a-4a68-b8dd-ab11ad3f9582 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.564034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.564683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.564963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.565237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.565919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.567782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.567782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:32:36.567782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:32:36.569026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:36.691697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 745/1497] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:32:36 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:32:36.695988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.705920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1b0a2-c7d7-45d0-a06d-447df18f60d9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.762846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.763496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.763775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.764045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.764721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.766352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.766352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:32:36.766352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:32:36.767642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:36.838249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 753/1498] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:32:36 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:32:36.842249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.853112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f127f2-4fe3-40d2-b3a6-50e593db15b1 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.857634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.858353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.858637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.858910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.859634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.860648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.860648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:32:36.860648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:32:36.861529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:36.874419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 746/1499] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:32:36.878811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.889127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9a697-1315-4ce7-859a-627ab158b83d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.894392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.895056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.895340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.895617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.896341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.897348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.897348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:32:36.897348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:32:36.898271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddce0b-72e0-4937-baba-68b410ed5e9f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:36.913042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 754/1500] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:32:36.917389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:36.927907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ce404-e2f7-4b92-99a1-46a4bc09dc35 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:36.932396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:36.933263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:36.933545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:36.933818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] PATH_INFO: `/v3/projects/a00e1e28b9544e849a49a6b707eb2054/users/a158727352324e6eb810f5f2df07b1c1/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:36.949882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00e1e28b9544e849a49a6b707eb2054, user_id=a158727352324e6eb810f5f2df07b1c1, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:36.950987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:36.950987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:32:36.950987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:32:36.952937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b2650-c76d-4f5a-872f-c114bd7022a1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:37.055465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 747/1501] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:32:36 2022] PUT /identity/v3/projects/a00e1e28b9544e849a49a6b707eb2054/users/a158727352324e6eb810f5f2df07b1c1/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:32:37.066939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29ea2-0043-4843-8e55-ba22682efe2f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.067224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29ea2-0043-4843-8e55-ba22682efe2f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.067496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e29ea2-0043-4843-8e55-ba22682efe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.094930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-97e29ea2-0043-4843-8e55-ba22682efe2f None None] MFA Rules not processed for user `a158727352324e6eb810f5f2df07b1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:32:37.096114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-97e29ea2-0043-4843-8e55-ba22682efe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:37.159322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 755/1502] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:32:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:32:37.172269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c775-4f03-4042-ae1b-243353188ef3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.172560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c775-4f03-4042-ae1b-243353188ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.172840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c775-4f03-4042-ae1b-243353188ef3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.174208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 748/1503] 10.222.0.48 () {54 vars in 963 bytes} [Fri May 27 09:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:32:37.179081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b03df-2f8f-40c1-8171-9df2f503ca26 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.179367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b03df-2f8f-40c1-8171-9df2f503ca26 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.179640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b03df-2f8f-40c1-8171-9df2f503ca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.198270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-883b03df-2f8f-40c1-8171-9df2f503ca26 None None] MFA Rules not processed for user `3963b8bacbc74ef2a579be684a219ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:32:37.199381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-883b03df-2f8f-40c1-8171-9df2f503ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:37.244666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:32:37.252803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 756/1504] 10.222.0.48 () {58 vars in 1060 bytes} [Fri May 27 09:32:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:32:37.262917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c775-4f03-4042-ae1b-243353188ef3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:37.272435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c775-4f03-4042-ae1b-243353188ef3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:37.279920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:37.280560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.280848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.281119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.290551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:37.291988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:37.302630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 749/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:32:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:32:37.348569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a044e-7fab-402f-b43a-7f293d6c3af7 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.348856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a044e-7fab-402f-b43a-7f293d6c3af7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.349133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a044e-7fab-402f-b43a-7f293d6c3af7 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.350479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 757/1506] 10.222.0.48 () {52 vars in 829 bytes} [Fri May 27 09:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:32:37.354291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:37.425156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179df85-0f31-4449-a0d4-7444407ed21c service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:37.429898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:37.430793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:37.431110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:37.431393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] PATH_INFO: `/v3/users/a158727352324e6eb810f5f2df07b1c1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:37.432625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a158727352324e6eb810f5f2df07b1c1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:37.433681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:37.433681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:32:37.433681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:32:37.435252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:37.911626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a158727352324e6eb810f5f2df07b1c1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:37.924076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a158727352324e6eb810f5f2df07b1c1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:37.944385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a158727352324e6eb810f5f2df07b1c1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:37.945211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a158727352324e6eb810f5f2df07b1c1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:37.949438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:32:38.121157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a158727352324e6eb810f5f2df07b1c1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:38.127559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 750/1507] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:32:37 2022] DELETE /identity/v3/users/a158727352324e6eb810f5f2df07b1c1 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:32:38.132184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:38.132687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:38.132791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:38.150669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:32:38.151878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:38.212502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 758/1508] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:32:38.222424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:38.223845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:38.283281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b822-36fe-412c-a62e-fb8b8eb7c2c7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:38.288522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:38.289167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:38.289445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:38.289714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:38.299134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:38.300569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e2aa0-754a-4429-bc03-44622cecb1c0 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:38.309989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 751/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:32:38.811410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:32:38.813442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:32:38.827240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259daa92-0671-4fad-830d-ea2ecd29a69b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:32:38.834549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:32:38.835458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:32:38.836222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:32:38.836505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] PATH_INFO: `/v3/projects/a00e1e28b9544e849a49a6b707eb2054` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:32:38.838116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00e1e28b9544e849a49a6b707eb2054)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:32:38.839186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:32:38.839186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:32:38.839186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:32:38.840409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:32:38.956563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00e1e28b9544e849a49a6b707eb2054', 'request_id': 'req-2643cba6-ce24-44c7-bbd8-709bdf6139ce'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:38.970162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00e1e28b9544e849a49a6b707eb2054', 'request_id': 'req-2643cba6-ce24-44c7-bbd8-709bdf6139ce'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:38.976161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00e1e28b9544e849a49a6b707eb2054', 'request_id': 'req-2643cba6-ce24-44c7-bbd8-709bdf6139ce'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:39.054375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] The token cache is being invalidate because project a00e1e28b9544e849a49a6b707eb2054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:32:39.054584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:39.055517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2643cba6-ce24-44c7-bbd8-709bdf6139ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:32:39.057188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 759/1510] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:32:38 2022] DELETE /identity/v3/projects/a00e1e28b9544e849a49a6b707eb2054 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:32:48.305666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:32:49.051009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:34:21.702633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:21.703407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:21.703407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:21.704708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:21.710370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8357ad2-87ce-4ad2-8d6f-94636232f6eb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:21.710797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8357ad2-87ce-4ad2-8d6f-94636232f6eb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:21.710929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8357ad2-87ce-4ad2-8d6f-94636232f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:21.729492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f8357ad2-87ce-4ad2-8d6f-94636232f6eb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:21.733208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f8357ad2-87ce-4ad2-8d6f-94636232f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:21.789908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:21.956934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:21.962364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:22.033402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00095-7abb-4980-bc2c-a997422930dc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:22.038790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:22.039435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:22.039718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:22.039996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:22.047956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:22.049555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf8ab0-d7c4-4485-bb0e-5231e0bbbc55 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:22.059599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 753/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:34:22.934574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac5e6-d017-4163-9387-2e0f744b77b3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:22.935119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac5e6-d017-4163-9387-2e0f744b77b3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:22.935119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfac5e6-d017-4163-9387-2e0f744b77b3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:22.936561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:22.941470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037083-22ce-4438-98b7-c5abf47795c4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:22.941929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037083-22ce-4438-98b7-c5abf47795c4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:22.942039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037083-22ce-4438-98b7-c5abf47795c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:22.960707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b2037083-22ce-4438-98b7-c5abf47795c4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:22.961906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b2037083-22ce-4438-98b7-c5abf47795c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:22.969927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:23.125821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfac5e6-d017-4163-9387-2e0f744b77b3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:23.140797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfac5e6-d017-4163-9387-2e0f744b77b3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:23.146167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:23.146823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:23.150451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:23.150451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:23.155823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:23.157418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3deed6-c256-407e-a55d-28f8281394d7 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:23.167250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 762/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:23.334259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6338e-c627-4ba7-afcb-65c34728d50e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:23.334549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6338e-c627-4ba7-afcb-65c34728d50e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:23.334827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6338e-c627-4ba7-afcb-65c34728d50e None None] PATH_INFO: `/v3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:23.336136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 755/1517] 10.222.0.16 () {54 vars in 925 bytes} [Fri May 27 09:34:23 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:34:23.342147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449650bf-e190-4074-9a63-85ebf4e713d3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:23.342542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449650bf-e190-4074-9a63-85ebf4e713d3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:23.342857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449650bf-e190-4074-9a63-85ebf4e713d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:23.361433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-449650bf-e190-4074-9a63-85ebf4e713d3 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:23.362697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-449650bf-e190-4074-9a63-85ebf4e713d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:23.426727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 763/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Fri May 27 09:34:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:27.575701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8fe2b-6dc2-404d-a65b-5cb91fac8af1 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:27.576142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8fe2b-6dc2-404d-a65b-5cb91fac8af1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:27.576269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8fe2b-6dc2-404d-a65b-5cb91fac8af1 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:27.577580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 756/1519] 10.222.0.48 () {54 vars in 960 bytes} [Fri May 27 09:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:27.583262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808d2bf-fac6-4f3d-8152-98336945f39f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:27.583764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808d2bf-fac6-4f3d-8152-98336945f39f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:27.583864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808d2bf-fac6-4f3d-8152-98336945f39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:27.602458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1808d2bf-fac6-4f3d-8152-98336945f39f None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:27.603675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1808d2bf-fac6-4f3d-8152-98336945f39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:27.611993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 764/1520] 10.222.0.48 () {58 vars in 1057 bytes} [Fri May 27 09:34:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:27.620163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8fe2b-6dc2-404d-a65b-5cb91fac8af1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:27.633693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8fe2b-6dc2-404d-a65b-5cb91fac8af1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:27.638976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:27.639601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:27.639877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:27.640147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:27.649018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:27.650631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec00f92-e6d6-491e-bc97-0c7d9b601065 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:27.660891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 757/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:34.442838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b569e8-50fc-4051-85bf-ba80afc393a5 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:34.443413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b569e8-50fc-4051-85bf-ba80afc393a5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:34.443413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b569e8-50fc-4051-85bf-ba80afc393a5 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:34.444856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:34.449825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3a19b-dae1-458f-b1d2-bda46180bf3b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:34.450339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3a19b-dae1-458f-b1d2-bda46180bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:34.450432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3a19b-dae1-458f-b1d2-bda46180bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:34.468388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e1a3a19b-dae1-458f-b1d2-bda46180bf3b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:34.469631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e1a3a19b-dae1-458f-b1d2-bda46180bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:34.477955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:34.630767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b569e8-50fc-4051-85bf-ba80afc393a5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:34.645403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b569e8-50fc-4051-85bf-ba80afc393a5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:34.650974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:34.651616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:34.651896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:34.652312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:34.661713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:34.663363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c0030-9443-4d79-8f89-e6609db64a4d service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:34.673369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 766/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:35.554270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70fe2-0979-49d9-8c80-d79f406b1cce None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:35.554881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70fe2-0979-49d9-8c80-d79f406b1cce None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:35.554881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70fe2-0979-49d9-8c80-d79f406b1cce None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:35.556383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 09:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:35.562389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed0ff7-90c7-409f-a18b-ac51b00d4a3e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:35.562936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed0ff7-90c7-409f-a18b-ac51b00d4a3e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:35.562936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed0ff7-90c7-409f-a18b-ac51b00d4a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:35.582086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b2ed0ff7-90c7-409f-a18b-ac51b00d4a3e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:35.583388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b2ed0ff7-90c7-409f-a18b-ac51b00d4a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:35.591143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 09:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:35.749730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d70fe2-0979-49d9-8c80-d79f406b1cce None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:35.765228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d70fe2-0979-49d9-8c80-d79f406b1cce None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:35.771098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:35.771763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:35.772047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:35.772702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:35.780866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:35.782504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a7d88-457c-4f58-8bd9-0ea297574ccf service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:35.792738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 760/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:40.045687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:40.046246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:40.046344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:40.047770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:40.054024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013525be-e826-4515-863c-8c34ed9abd42 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:40.054568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013525be-e826-4515-863c-8c34ed9abd42 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:40.054665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013525be-e826-4515-863c-8c34ed9abd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:40.073300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-013525be-e826-4515-863c-8c34ed9abd42 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:40.074529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-013525be-e826-4515-863c-8c34ed9abd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:40.082920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:40.120201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:40.122050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:40.187981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cd5a1-6532-43d5-a9b9-f526ea8879cf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:40.192679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:40.193377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:40.193745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:40.194028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:40.204429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:40.206030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931bcea-f5f8-4127-87d2-6a852e06a0c6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:40.212132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:34:40.217666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 769/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:42.398595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:42.399158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:42.399158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:42.400560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:42.405619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb7f1-d699-4764-941b-51d8be14784b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:42.406076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb7f1-d699-4764-941b-51d8be14784b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:42.406185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb7f1-d699-4764-941b-51d8be14784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:42.428860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a5dcb7f1-d699-4764-941b-51d8be14784b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:42.430035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5dcb7f1-d699-4764-941b-51d8be14784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:42.437952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:42.474146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:42.475835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:42.489441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2575-185f-490a-a888-336ae18d92c1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:42.492937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:42.493556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:42.493834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:42.494108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:42.503959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:42.505396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f457d75-7c60-4bc2-8ba8-0ae8f84cc09c service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:42.514948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 763/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:46.233874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:46.234492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:46.234492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:46.235838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:46.241274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3792e-8d22-4ae0-81e9-11a909fec73b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:46.241687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3792e-8d22-4ae0-81e9-11a909fec73b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:46.241818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3792e-8d22-4ae0-81e9-11a909fec73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:46.259589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b8e3792e-8d22-4ae0-81e9-11a909fec73b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:46.260807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b8e3792e-8d22-4ae0-81e9-11a909fec73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:46.268455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:46.537357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:46.539085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:46.597291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb5d46-0bad-4619-99a5-4411daa7dbee None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:46.600897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:46.601531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:46.601824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:46.602091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:46.611288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:46.612712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1270-7473-463a-8e6a-e25223ef77b1 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:46.621995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 772/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:48.856190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:48.856717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:48.856717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:48.858099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 09:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:34:48.863391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360583-dc1a-4583-86ba-a550bee194b3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:48.864896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360583-dc1a-4583-86ba-a550bee194b3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:48.864896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54360583-dc1a-4583-86ba-a550bee194b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:48.888074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-54360583-dc1a-4583-86ba-a550bee194b3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:34:48.888074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-54360583-dc1a-4583-86ba-a550bee194b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:48.895651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 09:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:34:49.168709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:34:49.170478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:34:49.178779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=65 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:34:49.185079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0e91-1121-4c2b-9572-821593f69cc0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:34:49.189562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:34:49.189562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:34:49.189562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:34:49.189904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:34:49.200603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:34:49.202042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b2ade-9eaa-45fd-b45e-848309104811 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:34:49.213183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 766/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:34:58.877115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=32 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:34:59.214303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:20.603187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.604269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.604772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.631919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:20.633513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:20.646354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 774/1540] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:20.662812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.663384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.663384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.691359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:20.692621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:20.706738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 767/1541] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:20.711398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:20.726617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7a05-9968-4cf6-a61f-a98773244c00 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:20.731911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:20.733007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.733299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.733576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.738603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:20.739015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:20.740738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8bdb-674e-4ff9-aca9-003812e651bf admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:20.765878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 775/1542] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:20.769416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:20.785587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5b4fb-8a64-47fd-8184-40b75967e26f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:20.799106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:20.800283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.801091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.801621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.802973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:20.805298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:20.876202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc854a-3088-49d0-899b-33a46d2584aa None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.876596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc854a-3088-49d0-899b-33a46d2584aa None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.876823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc854a-3088-49d0-899b-33a46d2584aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.903377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1ddc854a-3088-49d0-899b-33a46d2584aa None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:20.904628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1ddc854a-3088-49d0-899b-33a46d2584aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:20.916675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 768/1543] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:20.918129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 776/1544] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:20.921648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:20.927948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51f65-377d-446b-a545-4745ba321580 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:20.928256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51f65-377d-446b-a545-4745ba321580 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:20.928500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51f65-377d-446b-a545-4745ba321580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:20.959337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-34b51f65-377d-446b-a545-4745ba321580 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:20.961306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-34b51f65-377d-446b-a545-4745ba321580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.036800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707124-9d69-4353-95db-e84a85e55cab admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.048561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.049245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.049535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.049810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.050542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.051589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:21.051589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:36:21.051589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:36:21.052819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2af52-c617-4f41-9a9d-e9de1c749308 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.075949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 777/1545] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.080472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3306-138e-4cd5-be77-648b5ceaab13 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.080472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3306-138e-4cd5-be77-648b5ceaab13 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.080472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3306-138e-4cd5-be77-648b5ceaab13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.102532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e8cd3306-138e-4cd5-be77-648b5ceaab13 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.103834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e8cd3306-138e-4cd5-be77-648b5ceaab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.112478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 778/1546] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.113704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbf2b-4914-42ca-89b4-7a01a7d88820 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.113994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbf2b-4914-42ca-89b4-7a01a7d88820 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.114297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbf2b-4914-42ca-89b4-7a01a7d88820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.134475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d4adbf2b-4914-42ca-89b4-7a01a7d88820 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.135666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d4adbf2b-4914-42ca-89b4-7a01a7d88820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.144005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 779/1547] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.145346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.145628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.145895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.159710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 769/1548] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:20 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:21.163733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8561e6-f050-4d50-b2dd-1dce85025ca2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.163733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8561e6-f050-4d50-b2dd-1dce85025ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.163733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8561e6-f050-4d50-b2dd-1dce85025ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.164093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.165237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.173420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 780/1549] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.175208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.184346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-bb8561e6-f050-4d50-b2dd-1dce85025ca2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.185590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bb8561e6-f050-4d50-b2dd-1dce85025ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.189992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f09d7-565f-48b1-8170-a8fb7cfaf6db None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.194559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 770/1550] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.195698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa979-8c79-430a-a9f1-d738d9691999 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.195991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa979-8c79-430a-a9f1-d738d9691999 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.196216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.196351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa979-8c79-430a-a9f1-d738d9691999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.197161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.197631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.198084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.199292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.200750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.201926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbcdc7-4bf0-4eec-a0b6-a7a5b7257a54 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.219260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 781/1551] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.220680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e83da-e213-41d7-bc1a-579bfb95bc9e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.220971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e83da-e213-41d7-bc1a-579bfb95bc9e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.221247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e83da-e213-41d7-bc1a-579bfb95bc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.224202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-814aa979-8c79-430a-a9f1-d738d9691999 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.225556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-814aa979-8c79-430a-a9f1-d738d9691999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.233423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 771/1552] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.235342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.235342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.235342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.245472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6b2e83da-e213-41d7-bc1a-579bfb95bc9e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.246793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6b2e83da-e213-41d7-bc1a-579bfb95bc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.254589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.255760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.258143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 782/1553] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.258143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b504a-ca38-424c-a6fb-c2c77ce4fa91 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.258143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b504a-ca38-424c-a6fb-c2c77ce4fa91 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.265718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b504a-ca38-424c-a6fb-c2c77ce4fa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.265820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 772/1554] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.269487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.277826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-957b504a-ca38-424c-a6fb-c2c77ce4fa91 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.279135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc09b8-78f2-4d36-b2d0-c5a42c01efac None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.279278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-957b504a-ca38-424c-a6fb-c2c77ce4fa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.283350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.286926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.286926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.286926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.286926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.286926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fbba3-5aaf-4281-a164-0b4d2f540570 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.287473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 783/1555] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.288737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.289024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.289299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.302573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 773/1556] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:21.304069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88132baa-34f1-4131-bffa-d31280ed89d1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.304413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88132baa-34f1-4131-bffa-d31280ed89d1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.304751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88132baa-34f1-4131-bffa-d31280ed89d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.308035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.309272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.318763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 784/1557] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.319122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.324945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-88132baa-34f1-4131-bffa-d31280ed89d1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.327118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-88132baa-34f1-4131-bffa-d31280ed89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.328534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec045a-b5d3-471b-aac8-e76e145b8cde None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.333105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.333755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.334034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 774/1558] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.334171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.334363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.335053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.335415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e4d05-867d-46ad-90d1-575d60228ccb None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.335756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e4d05-867d-46ad-90d1-575d60228ccb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.335974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e4d05-867d-46ad-90d1-575d60228ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.336089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:21.336089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:36:21.336089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:36:21.337266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.353913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b14e4d05-867d-46ad-90d1-575d60228ccb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.355791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b14e4d05-867d-46ad-90d1-575d60228ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.365762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 775/1559] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.367217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.367480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.367750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.386442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:21.387624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:21.395382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 776/1560] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:21.396138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.405181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8cf7-0481-44fd-a09a-ee931e9ce254 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.409831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.410524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.410898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.411271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.411989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.412296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.413338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:21.413338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:36:21.413338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:36:21.414222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.427152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 777/1561] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.427620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.437309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdc07c-b8ad-400c-b0c6-73c9a9f49731 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.442426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.445304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.445304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.445304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.445304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.445304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.446153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.457915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 778/1562] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.458887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.468581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94c3b3-50d4-461e-beee-ee14adea48c1 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.473207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.473861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.474151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.474461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.475191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.476768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.487347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 779/1563] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:21.490571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.497973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 785/1564] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:21.498124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13c198-e75a-4ab7-8574-24f0121470ad admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.498797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.529016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:21.557529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.558213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.558507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.558786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.559513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.559820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.561449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.563085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1253a-17ca-49db-8e34-9cd28c612388 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.568462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.569179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.569461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.569734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.570763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.571073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.572680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.574212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 780/1565] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.574651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:21.575178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.585893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22c1a9-cdcc-4802-8486-d4bc97428f41 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.594262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 786/1566] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.594262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.597971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.597971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.597971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.597971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.598384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.598384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.599562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.604076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa88f8-0de5-4449-8316-cddb089c8868 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.608739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.609385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.609659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.610513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.611234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:21.611534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.619776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 781/1567] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.619985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.620091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.626401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 787/1568] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:21.629971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689329-1554-469e-972f-eb5b8abc58ed admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.631180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.634943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.635682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.636134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.636412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.637123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.639047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.642150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74dc18-02c1-418e-a68e-75cd337c6219 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.647120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.647735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.648046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.648312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.649346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.651783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.752537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 782/1569] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:21.753366 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.775115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 788/1570] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:21.776363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.821230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5b7d1-5987-4621-9060-3cc6b09fd2e4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.841104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd416f-0663-454d-93b9-2b09b8ae53c5 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.846707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.847450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.847744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.848017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.848709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.850737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:21.939995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 789/1571] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:21.939995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:21.951741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c9b1a-33b1-4f37-9c32-7ad247dbf70f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:21.956749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:21.957405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:21.957689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:21.958037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:21.958719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:21.960892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.007864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.008654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.009039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.009471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] PATH_INFO: `/v3/projects/56f6fbec499344859b9da684d29ffdb8/users/2ca8c6589d0a4bbb9c5adf780cbd2fd8/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.023648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f6fbec499344859b9da684d29ffdb8, user_id=2ca8c6589d0a4bbb9c5adf780cbd2fd8, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.026501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.089365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 790/1572] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:21 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.090567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.122280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 783/1573] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:21 2022] PUT /identity/v3/projects/56f6fbec499344859b9da684d29ffdb8/users/2ca8c6589d0a4bbb9c5adf780cbd2fd8/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:22.123155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.149843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:22.153534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425ffdd-13a8-4364-8343-7699452a7c8e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.184273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:22.190641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c4344-06b4-4afd-b4f5-c22ad73b881c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.195379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.196037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.196317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.196588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.197311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.199285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.210612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.211268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.211975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.211975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.212496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.214457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.319055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 784/1574] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.320228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.331067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 791/1575] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.331067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.388381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a65d-def1-4de8-9e53-b8dd0d96d842 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.392706 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1267-5033-4503-9190-a13a74ea10fd admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.393673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.394371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.394655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.394931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.395621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.397544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.402449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.402449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.402449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.402449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.402889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.403111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.495319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 785/1576] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.496187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.503268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 792/1577] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.504301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.516825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda84aa-e3a7-4750-9920-9bb327d1d377 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.566231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b439b-fee4-4fe9-be29-0c512abafca5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.571007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.571683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.571969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.572250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.572941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.574926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b0bfc-94dd-4566-aee7-2cdfde9788c3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.576787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.577446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.577749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.578016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.579332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.581323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.591659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 793/1578] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:22.596133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.602648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8e4f8-e102-414d-8787-9a4d2bf411f3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.607224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.607928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.608209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.608480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.609156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.611110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.642746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 786/1579] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.644067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.644349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.644622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.661410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:22.662748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:22.670129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 787/1580] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:22.670894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.679990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8185-9417-4058-bb80-431f4eeed56b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.684786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.690476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.690476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.690476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.690476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.690476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.713597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 794/1581] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.714513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.723976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3a101-598b-4c4d-bd90-f288904123fd admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.728515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.729188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.729461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.729728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.730434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.732309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.800057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:22.806453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 788/1582] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.807378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.817691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca7ded-4438-42fa-bdd0-342204121610 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.822754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.823403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.823680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.823970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.824675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.826744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.867385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 795/1583] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.867385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.877723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbacdbe-9bed-4b5a-9a20-4e8529ed9b5b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.882915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.883640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.883961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.884350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.885138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.886991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.889389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:22.898288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 796/1584] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:22.899368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.915951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccea6-2af5-4fc9-8b2f-563bfb0bbc84 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.921276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.921937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.922244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.923049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.923776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.925367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.936169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 797/1585] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:22.937314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.946839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0926-ed8b-446d-9111-cf5672a971a2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.951694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.952339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.952620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.952896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.953659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.955354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:22.963852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 789/1586] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:22 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:22.963852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.969457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 798/1587] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:22.970485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:22.981455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a64199-2193-42cf-bb31-f8e2ae3e843b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:22.986970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:22.987631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:22.987912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:22.988220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:22.989019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:22.991106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.003613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 799/1588] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.004544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.015913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a3b9c-88b4-414f-b02a-27939575d50f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.020959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.021656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.021943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.022243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.022838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4c86-0fc6-46bf-b61b-5c70828d397c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.023026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.024615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.026615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.027289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.027569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.027858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.034921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 800/1589] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.035730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.043212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.045206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.045865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d55f9-e621-4f8c-84fe-294f71a45226 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.051333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.052330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.052637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.053994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.053994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.055214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.055682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 790/1590] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:23.056856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.067313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c41def-81da-45a1-91df-c58e77b8a5f1 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.068801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 801/1591] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.069622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.080200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.080200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.080200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.080200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.080691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.080691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.080864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34b1d-ea01-46ea-bed0-9bb5b0af8e02 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.086819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.087878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.088171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.088438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] PATH_INFO: `/v3/projects/817a1b039baf44bdab5e6730f97a7f9f/users/db9cf1d3db2140d58551f86c84e2b3db/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.091036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 791/1592] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.095167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.104541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=817a1b039baf44bdab5e6730f97a7f9f, user_id=db9cf1d3db2140d58551f86c84e2b3db, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.105081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f94dc-96b5-46d1-8d7f-48039dfb8d68 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.105780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:23.105780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:36:23.105780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:36:23.108030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.110543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.111209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.111601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.111873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.112593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.114281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.137006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 792/1593] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.137940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.147942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0d43c-1279-4f17-9c4e-100361e82bdd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.154849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.154849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.154849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.154849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.155403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.157072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.169284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 793/1594] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.169929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.181165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dfd40-be21-485e-a5bd-7dbb20afddc9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.186148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.186840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.187125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.187403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.188677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.190307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.201371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 794/1595] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.203256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.208760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 802/1596] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/817a1b039baf44bdab5e6730f97a7f9f/users/db9cf1d3db2140d58551f86c84e2b3db/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.209216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.224153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe2c9-a09b-4930-ba09-85085d401e0b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.279584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add14a8-cecc-4573-9b48-944cb5290921 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.285293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.285973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.286308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.286663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.287616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.289536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.299227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.299895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.300177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.300454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 803/1597] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.300553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.301268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.301384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.302944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.313756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 795/1598] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.314913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.317045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e2f8-af4c-4c1f-aee9-83e008087e1f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.323596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.324487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.324839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.324943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79ba16-0f77-4738-989e-bc70f267a318 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.325062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] PATH_INFO: `/v3/projects/64c171ba5fcb489d911aeb8c5abf4bb2/users/e5081aed86004f6c99a05c9fbf2a36b9/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.330544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.331314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.331601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.331708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.332387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.333992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.343205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c171ba5fcb489d911aeb8c5abf4bb2, user_id=e5081aed86004f6c99a05c9fbf2a36b9, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.346497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.351696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 796/1599] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:23.351696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.361602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de45794-5c02-4ea9-a8de-a8a945a77b44 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.366753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.367592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.367862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.368132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] PATH_INFO: `/v3/projects/e10012e947154b95b38fe47f16c88ab6/users/66bc9b7cbe98440ea5cc5270ae6973ec/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.383873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10012e947154b95b38fe47f16c88ab6, user_id=66bc9b7cbe98440ea5cc5270ae6973ec, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.387745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280aa0e-7f7d-4b3a-8bad-fa8f34c9fc39 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.425742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 804/1600] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/64c171ba5fcb489d911aeb8c5abf4bb2/users/e5081aed86004f6c99a05c9fbf2a36b9/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.430989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.481439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:23.483723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:23.489293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 797/1601] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/e10012e947154b95b38fe47f16c88ab6/users/66bc9b7cbe98440ea5cc5270ae6973ec/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.490788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.491074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.491338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.494603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f5e5f-c03c-4d0d-ba73-6b801ec891be admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.522516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] MFA Rules not processed for user `db9cf1d3db2140d58551f86c84e2b3db`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:23.523730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:23.556275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.557366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.557672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.558076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] PATH_INFO: `/v3/projects/9015a8cf2a9040fba7060e4d377d99ed/users/2ad1ad3922d4431391c938db27cbc280/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.576550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015a8cf2a9040fba7060e4d377d99ed, user_id=2ad1ad3922d4431391c938db27cbc280, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.579972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.596992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 798/1602] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:23.597578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.608585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755c67-36d1-45da-9c5b-094b193d035f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.613220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.614093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.614416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.614683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] PATH_INFO: `/v3/projects/1598c6c82b11451fa6851486e117d6f4/users/32ccc18481774bdcaf28dedf0edbd56c/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.629414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1598c6c82b11451fa6851486e117d6f4, user_id=32ccc18481774bdcaf28dedf0edbd56c, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.632788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.665932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 805/1603] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/9015a8cf2a9040fba7060e4d377d99ed/users/2ad1ad3922d4431391c938db27cbc280/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.666742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.738929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f87e8-0916-4e11-83f7-dccab7b441a4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.743619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.744513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.744776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.745041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] PATH_INFO: `/v3/projects/836f9646586d4f409c181cdbfa24cb32/users/8d52fe5018e2450dad79596fb1129c64/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.761091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] RBAC: Authorizing `identity:create_grant(project_id=836f9646586d4f409c181cdbfa24cb32, user_id=8d52fe5018e2450dad79596fb1129c64, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.763824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c518ef-8fdb-4432-97cd-9c496574efff admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.787048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 799/1604] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/1598c6c82b11451fa6851486e117d6f4/users/32ccc18481774bdcaf28dedf0edbd56c/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.788024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:23.844076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 806/1605] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/836f9646586d4f409c181cdbfa24cb32/users/8d52fe5018e2450dad79596fb1129c64/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:23.846256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157344c-3ea3-4701-b9f3-311441b2f1d5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.846582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157344c-3ea3-4701-b9f3-311441b2f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.846927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157344c-3ea3-4701-b9f3-311441b2f1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.875278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0157344c-3ea3-4701-b9f3-311441b2f1d5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:23.875278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0157344c-3ea3-4701-b9f3-311441b2f1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:23.875589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d51c8-293b-401e-92c5-85c72aa968e3 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:23.943215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:23.944111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.944388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.944660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] PATH_INFO: `/v3/projects/1ae983f1ecde4623986967c0ef40c295/users/3575512e61e24d14b802cd58542ffade/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.955687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 807/1606] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:23.957969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:23.959056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:23.960279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:23.961901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae983f1ecde4623986967c0ef40c295, user_id=3575512e61e24d14b802cd58542ffade, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:23.964757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54dab-9310-40a8-a44c-89b11c4844a5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:23.969623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:23.970036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:23.989693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] MFA Rules not processed for user `66bc9b7cbe98440ea5cc5270ae6973ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:23.990916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.057466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 808/1607] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:24.058370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 800/1608] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:23 2022] PUT /identity/v3/projects/1ae983f1ecde4623986967c0ef40c295/users/3575512e61e24d14b802cd58542ffade/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:24.062364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.072272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91625e5-98fe-4994-9a80-45e48aaa9909 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.072591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91625e5-98fe-4994-9a80-45e48aaa9909 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.072828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91625e5-98fe-4994-9a80-45e48aaa9909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.081997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d39a-8586-4f4f-ab50-ec0a013a51a0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.102534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d91625e5-98fe-4994-9a80-45e48aaa9909 None None] MFA Rules not processed for user `2ad1ad3922d4431391c938db27cbc280`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:24.103938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d91625e5-98fe-4994-9a80-45e48aaa9909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.146268 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.146854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.147140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.147419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.148118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.150348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9de49-893d-4e57-bbcb-6853fea7d815 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.171220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 801/1609] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:24.173315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.173596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.173889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.208079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] MFA Rules not processed for user `32ccc18481774bdcaf28dedf0edbd56c`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:24.209391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.236924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 809/1610] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/projects => generated 403 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:24.238409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81be3f-2eac-4a80-a13a-ef56c92d4242 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.238657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81be3f-2eac-4a80-a13a-ef56c92d4242 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.238932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81be3f-2eac-4a80-a13a-ef56c92d4242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.274837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7e81be3f-2eac-4a80-a13a-ef56c92d4242 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:24.276063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7e81be3f-2eac-4a80-a13a-ef56c92d4242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.311081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 802/1611] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:24.322240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.371322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 810/1612] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:24.373005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.373379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.374361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.417416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] MFA Rules not processed for user `3575512e61e24d14b802cd58542ffade`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:24.419844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.442218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:24.453540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e326a-0060-40f9-a25f-c3104fc5207c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.492691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:24.523326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.524149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.524557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.524994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.548275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.550574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.552071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 811/1613] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:24.553545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.593586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 803/1614] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:24.595043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.630250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13a7e5-6df7-4025-973d-46b538aca682 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.633919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.634662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.634890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.635156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.635861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.637956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.696353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2bffc-e9b4-438f-8c83-e6c571ee432d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.700075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.700724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.700999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.701276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.751754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 804/1615] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:24.756083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.760908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.763011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee1fae-fb29-49ec-a46e-8b032d68df6d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.772781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 812/1616] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:24.784731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2816-a824-48c6-860e-9a56fdf1cd51 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.785025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2816-a824-48c6-860e-9a56fdf1cd51 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.785415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2816-a824-48c6-860e-9a56fdf1cd51 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.795072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 813/1617] 10.222.0.48 () {54 vars in 961 bytes} [Fri May 27 09:36:24 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:24.796618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2816-a824-48c6-860e-9a56fdf1cd51 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.847581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc915ffb-cd62-4eb4-9d26-dc0d35092cc0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.852793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.853505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.853784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.854059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.855629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.857993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aeed9-563a-4a7a-8349-5c9534d32376 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.874732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2816-a824-48c6-860e-9a56fdf1cd51 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.878819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.879490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.879866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.880146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.891852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.893660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638d940-d527-4b58-9d35-6b438d800270 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:24.905747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 814/1618] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:24.907661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b2854-deee-4de5-a7cd-4d92a4cac4c5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.907986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b2854-deee-4de5-a7cd-4d92a4cac4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.908323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b2854-deee-4de5-a7cd-4d92a4cac4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.956781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ed2b2854-deee-4de5-a7cd-4d92a4cac4c5 None None] MFA Rules not processed for user `2ca8c6589d0a4bbb9c5adf780cbd2fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:24.958968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ed2b2854-deee-4de5-a7cd-4d92a4cac4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:24.963926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 805/1619] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/users => generated 308 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:24.963926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572ee5-2ff1-46a9-a124-0bc3fae810bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.963926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572ee5-2ff1-46a9-a124-0bc3fae810bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.963926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572ee5-2ff1-46a9-a124-0bc3fae810bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.963926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 806/1620] 10.222.0.48 () {54 vars in 961 bytes} [Fri May 27 09:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:24.964708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572ee5-2ff1-46a9-a124-0bc3fae810bf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:24.979785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572ee5-2ff1-46a9-a124-0bc3fae810bf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:24.986506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:24.987380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:24.987696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:24.988070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:24.988925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:24.991409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.059318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 815/1621] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:25.060932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.061224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.064176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.084017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:25.090911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:25.091222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 807/1622] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:24 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:25.091222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.103653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7ca1d-33e9-42c2-b11b-e4bb0c2961fd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.109785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.115124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.115124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.115124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.115124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.115499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:25.115608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849491b-7e89-4e31-8a75-87a7beb2cfcc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.134408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 808/1623] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:25.136732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7578-6dab-4301-8805-d56f252616f2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.137195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7578-6dab-4301-8805-d56f252616f2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.137644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7578-6dab-4301-8805-d56f252616f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.167103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2afb7578-6dab-4301-8805-d56f252616f2 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:25.168474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2afb7578-6dab-4301-8805-d56f252616f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:25.179480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 809/1624] 10.222.0.48 () {58 vars in 1058 bytes} [Fri May 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:25.181310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.181749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.181878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 816/1625] 10.222.0.48 () {58 vars in 1058 bytes} [Fri May 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:25.182129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.182752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.203139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] MFA Rules not processed for user `2ca8c6589d0a4bbb9c5adf780cbd2fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:25.205082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:25.208994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c74a-ede6-45cb-91cb-7316d0bd0751 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.213711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 810/1626] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:25.214809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.223013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.223757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.224038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.224310 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.225089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.227010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.236126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bab01-5c0b-4b66-8b75-5348d7183e28 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.242246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.242703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 817/1627] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:25.243722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.243887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.244238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.244769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] PATH_INFO: `/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9/users/35f5b5524c2f4101862d4e53759f0e2c/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.273266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa76c-84d8-47da-b163-507e88795b30 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.273651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ef2ed6c6d54cd0b841ee9eb704d0e9, user_id=35f5b5524c2f4101862d4e53759f0e2c, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.273651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.273850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.274180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.274491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.275031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.369346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 811/1628] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:25 2022] PUT /identity/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9/users/35f5b5524c2f4101862d4e53759f0e2c/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:25.370885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.371469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.373753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.394645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 818/1629] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:25.397998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.415439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ff47-51a9-4733-9dfc-5893f3602895 service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.435077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:25.436453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:25.475197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963a56-6c2b-48b9-a9be-325a1e322f5f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.483274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.484112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.484390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.484660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.515399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.524979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.525610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.526227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] PATH_INFO: `/v3/projects/5bb8f316d0984031bfa9c0062d2a1672/users/dad9e02337e340f3a244ef78f0f89e31/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.555273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb8f316d0984031bfa9c0062d2a1672, user_id=dad9e02337e340f3a244ef78f0f89e31, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.560087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.571393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.573397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.585335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 812/1630] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:25.586078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.603914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd21a0-409c-424b-b76a-fbcc8c64a423 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.609805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.611054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.611334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.611603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.613093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.615407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.632487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 813/1631] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:25.633482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.646955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15810f-5cbd-4043-8f17-7303712706e5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.650247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 819/1632] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:25 2022] PUT /identity/v3/projects/5bb8f316d0984031bfa9c0062d2a1672/users/dad9e02337e340f3a244ef78f0f89e31/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:25.653891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:25.771660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.772696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.773094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.773230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] PATH_INFO: `/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9/users/35f5b5524c2f4101862d4e53759f0e2c/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.778054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ef2ed6c6d54cd0b841ee9eb704d0e9, user_id=35f5b5524c2f4101862d4e53759f0e2c, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.782864 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33405bd-59d1-4ab6-b847-5622a7eaef49 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.815612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 814/1633] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:25 2022] PUT /identity/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9/users/35f5b5524c2f4101862d4e53759f0e2c/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:25.817657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985994f7-3f90-4099-bfc1-accd45557b9e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.817958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985994f7-3f90-4099-bfc1-accd45557b9e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.818264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985994f7-3f90-4099-bfc1-accd45557b9e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.820322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 815/1634] 10.222.0.48 () {54 vars in 912 bytes} [Fri May 27 09:36:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:25.821306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.821669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.821955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.833234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823963-cf5e-42ae-b30b-608c78125e63 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:25.871431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] MFA Rules not processed for user `35f5b5524c2f4101862d4e53759f0e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:25.872937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:25.918371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:25.919405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.919695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.920009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.920926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:25.923162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51a31d-1d59-4520-9b76-7bba1646ba4b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:25.938248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 820/1635] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:36:25.939071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b728e-4d69-4b40-b84b-2e4c02bc2bef None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:25.942911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b728e-4d69-4b40-b84b-2e4c02bc2bef None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:25.942911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b728e-4d69-4b40-b84b-2e4c02bc2bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:25.964331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1e3b728e-4d69-4b40-b84b-2e4c02bc2bef None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:25.966099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1e3b728e-4d69-4b40-b84b-2e4c02bc2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:26.021481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:26.022457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 816/1636] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:26.024470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:26.043948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:26.045545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b21918-2447-48a2-86e4-e7aeb2ffb919 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:26.052350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:26.056944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.056944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.056944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] PATH_INFO: `/v3/projects/5bb8f316d0984031bfa9c0062d2a1672/users/dad9e02337e340f3a244ef78f0f89e31/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.057478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb8f316d0984031bfa9c0062d2a1672, user_id=dad9e02337e340f3a244ef78f0f89e31, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:26.061891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20382f45-166f-48e0-b4b8-622bacddeee6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 817/1637] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:26 2022] PUT /identity/v3/projects/5bb8f316d0984031bfa9c0062d2a1672/users/dad9e02337e340f3a244ef78f0f89e31/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f971a-4dda-461a-a5d5-cc322cf776d3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f971a-4dda-461a-a5d5-cc322cf776d3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f971a-4dda-461a-a5d5-cc322cf776d3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 818/1638] 10.222.0.48 () {54 vars in 912 bytes} [Fri May 27 09:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:26.090701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f971a-4dda-461a-a5d5-cc322cf776d3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:26.103211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 821/1639] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:26.105772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a064a-3dff-44ac-966e-b3fc3f477ef7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.106054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a064a-3dff-44ac-966e-b3fc3f477ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.106472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a064a-3dff-44ac-966e-b3fc3f477ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.135700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4c2a064a-3dff-44ac-966e-b3fc3f477ef7 None None] MFA Rules not processed for user `dad9e02337e340f3a244ef78f0f89e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:26.136912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4c2a064a-3dff-44ac-966e-b3fc3f477ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:26.175763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f971a-4dda-461a-a5d5-cc322cf776d3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:26.179550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:26.180194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.180498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.181005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.226403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 822/1640] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:26.228226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.228732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.229110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.254578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:26.256704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:26.273933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:26.275592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:26.289333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 819/1641] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:26.290473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:26.301687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed948-ac77-4d1e-a5a2-1b07ea5cf88d service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:26.315722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:26.316533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.316912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.317285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.372846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 823/1642] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:26.420323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:26.434135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a40d-703b-4674-846f-467affb471e5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:26.436500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:26.438401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:26.438735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:26.443646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.443646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.443646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.452465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:26.452976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 820/1643] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:26.454226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:26.468750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 824/1644] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:36:26.628893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:26.633104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:26.638412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03698caa-5ca4-4859-ae15-113fce70eceb service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:26.644271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:26.644271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:26.644271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:26.644271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:26.654843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:26.656533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:26.670162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 821/1645] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:27.958988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:27.992072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:28.004399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c664757-99f1-4590-aab3-acb7398fecf2 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:28.020262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbe78f-6a61-4051-9b2c-9a19bef12d08 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:28.036549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:28.037214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.037496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.037769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.042875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:28.043908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.044191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.044465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.136397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:28.194711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:28.199522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:28.205264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:28.206956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9fd3c-39f0-44e3-a308-fe6c3f0d27aa service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:28.220959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 822/1646] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:36:28.222337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:28.228884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 825/1647] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:28.439263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a474d-961b-41c6-840f-f68dd5995204 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:28.461070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:28.461070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.461070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.461381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] PATH_INFO: `/v3/users/db9cf1d3db2140d58551f86c84e2b3db` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.468040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9cf1d3db2140d58551f86c84e2b3db)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:28.475897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:28.695568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9cf1d3db2140d58551f86c84e2b3db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:28.705640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9cf1d3db2140d58551f86c84e2b3db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:28.721946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9cf1d3db2140d58551f86c84e2b3db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:28.723532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9cf1d3db2140d58551f86c84e2b3db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:28.836646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-82291c24-f4cb-4c9b-9d8e-4f1378cd2600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9cf1d3db2140d58551f86c84e2b3db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:28.847096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 823/1648] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:28 2022] DELETE /identity/v3/users/db9cf1d3db2140d58551f86c84e2b3db => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:28.848896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.849360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.849597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.857612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08222f1-d72e-4dfa-b7bb-c93de8325904 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.857897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08222f1-d72e-4dfa-b7bb-c93de8325904 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.858171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08222f1-d72e-4dfa-b7bb-c93de8325904 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.859583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 824/1649] 10.222.0.48 () {54 vars in 912 bytes} [Fri May 27 09:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:36:28.865488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:28.866268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:28.866621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:28.880712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:28.882100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:28.903030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:28.959740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:28.960564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:29.035320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 826/1650] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:29.056084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:29.056084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:29.066418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 825/1651] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:29.088220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2592e-6ced-4d1a-a2ca-4677cf9fb77d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:29.093917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:29.095047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:29.095340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:29.095705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:29.108691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:29.109112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:29.115211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:29.116861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e4d97-6d52-40cd-9395-f77422e327fc service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:29.133392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 827/1652] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:36:29.134299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7954d2-7d02-432a-886a-f3102db67835 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:29.144911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:29.146315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:29.148405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:29.148786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:29.161958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:29.164743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0a550-a4f3-4071-94c6-794301f34e51 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:29.175310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 826/1653] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:30.318402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d356-6a63-413c-822e-710d57c53a2c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:30.318906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d356-6a63-413c-822e-710d57c53a2c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:30.319078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173d356-6a63-413c-822e-710d57c53a2c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:30.320609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 828/1654] 10.222.0.48 () {54 vars in 960 bytes} [Fri May 27 09:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:30.331425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:30.332927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:30.333414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:30.374957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:30.376891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:30.620402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 827/1655] 10.222.0.48 () {58 vars in 1057 bytes} [Fri May 27 09:36:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:30.632521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173d356-6a63-413c-822e-710d57c53a2c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:30.655149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173d356-6a63-413c-822e-710d57c53a2c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:30.668326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:30.669074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:30.669493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:30.669751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:30.673461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:30.675156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:30.683861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:30.719851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83792578-d21a-48a1-95a2-df4260710a39 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:30.731758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:30.732455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:30.733590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:30.735502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:30.736325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:30.736714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] PATH_INFO: `/v3/projects/817a1b039baf44bdab5e6730f97a7f9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:30.740363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=817a1b039baf44bdab5e6730f97a7f9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:30.741482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:30.741482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: warnings.warn(msg) May 27 09:36:30.741482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: May 27 09:36:30.742846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:30.760509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 829/1656] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:36:30.987060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817a1b039baf44bdab5e6730f97a7f9f', 'request_id': 'req-4ec53bf1-c798-40f7-92cc-89405ce26f79'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.045173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817a1b039baf44bdab5e6730f97a7f9f', 'request_id': 'req-4ec53bf1-c798-40f7-92cc-89405ce26f79'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.074245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817a1b039baf44bdab5e6730f97a7f9f', 'request_id': 'req-4ec53bf1-c798-40f7-92cc-89405ce26f79'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.223252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] The token cache is being invalidate because project 817a1b039baf44bdab5e6730f97a7f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:31.223252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.227418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.227418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 828/1657] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:30 2022] DELETE /identity/v3/projects/817a1b039baf44bdab5e6730f97a7f9f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:31.231088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:31.249771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bec71-b313-4045-9bc3-06061ca497c9 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:31.417207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:31.418600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:31.418949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:31.419878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] PATH_INFO: `/v3/users/35f5b5524c2f4101862d4e53759f0e2c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:31.428509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f5b5524c2f4101862d4e53759f0e2c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:31.428509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 09:36:31.428509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: warnings.warn(msg) May 27 09:36:31.428509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: May 27 09:36:31.436574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:31.851486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f5b5524c2f4101862d4e53759f0e2c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.853519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f5b5524c2f4101862d4e53759f0e2c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.876281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:31.948650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f5b5524c2f4101862d4e53759f0e2c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:31.952791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f5b5524c2f4101862d4e53759f0e2c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:32.017037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-13f5743b-5cc2-4abb-b9f8-eb4247370fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f5b5524c2f4101862d4e53759f0e2c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:32.105969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 830/1658] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:31 2022] DELETE /identity/v3/users/35f5b5524c2f4101862d4e53759f0e2c => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:33.167466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:33.175188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:33.195017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53bf1-c798-40f7-92cc-89405ce26f79 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:33.211916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:33.336429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:33.337760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:33.338185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:33.338626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] PATH_INFO: `/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:33.340247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ef2ed6c6d54cd0b841ee9eb704d0e9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:33.342416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:33.509003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ef2ed6c6d54cd0b841ee9eb704d0e9', 'request_id': 'req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:33.517623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ef2ed6c6d54cd0b841ee9eb704d0e9', 'request_id': 'req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:33.525093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ef2ed6c6d54cd0b841ee9eb704d0e9', 'request_id': 'req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:33.626677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] The token cache is being invalidate because project 19ef2ed6c6d54cd0b841ee9eb704d0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:33.627196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:33.628887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5d8e30cf-2f24-476e-a91e-ca88d4bcd438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:33.634601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 829/1659] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:33 2022] DELETE /identity/v3/projects/19ef2ed6c6d54cd0b841ee9eb704d0e9 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:33.639990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:33.641460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:33.641879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:33.702087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:33.704214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:33.926085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 831/1660] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:33.934669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:33.934961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:33.935231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:33.973230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:33.977519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:33.993132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 830/1661] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:33.997626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.016826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048ead0-2bcb-4377-afba-27ac153e9429 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.027144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.027919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.028174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.028355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.034101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:34.040817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.043156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc082bec-6bfe-4832-b95e-ae134bc0b354 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.085446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 832/1662] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:34.086649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.098254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ca236-dfbf-4eb8-ac57-678570594795 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.111821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.113495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.113495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.113495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.113829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.117017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.228873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8cc3-e078-4b41-a33d-45dbe0aa27d9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.229162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8cc3-e078-4b41-a33d-45dbe0aa27d9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.229475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8cc3-e078-4b41-a33d-45dbe0aa27d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.264324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cd8f8cc3-e078-4b41-a33d-45dbe0aa27d9 None None] MFA Rules not processed for user `e5081aed86004f6c99a05c9fbf2a36b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:34.265693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cd8f8cc3-e078-4b41-a33d-45dbe0aa27d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:34.287225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 831/1663] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:34 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:36:34.291762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.425307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb63e3-48aa-4771-b86a-df91383fc728 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.437081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.438872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.439608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.440073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.441152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.444760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.536331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 833/1664] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:34.538693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.539161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.539914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.540898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:34.547685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 832/1665] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:34 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:34.553637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.584389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0b1b-b3e3-491d-a091-07fec6734da9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.594003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] MFA Rules not processed for user `8d52fe5018e2450dad79596fb1129c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:34.600729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.600729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.601001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:34.601086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.601086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.601086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.601086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.616594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 833/1666] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:34.617283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.623454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:34.645947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:34.683587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 834/1667] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:34.684907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.704387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d532391-cb8b-4a13-8a38-5c25b80924cf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.704387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.704387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.704387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.704709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.704895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.706589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.717812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 835/1668] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:34.721236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.724998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:34.727846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeea9a4-2bf8-449c-baf8-f4774f13a66a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.739222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.740138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.740610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.741025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.747153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7592d-6ca0-407b-8210-58f041bd11c0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.754396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.755059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.755338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.755611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.765853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.771598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.779231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 836/1669] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:34.780107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:34.792993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789512e8-f923-4a44-b207-2c2483090f2d service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:34.800211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:34.801437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.801765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.802059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] PATH_INFO: `/v3/projects/6f6ac69be54148afa3df243ca3b2bd9c/users/aa054f202b914d45ac67e8c7737d01f8/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:34.824233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6ac69be54148afa3df243ca3b2bd9c, user_id=aa054f202b914d45ac67e8c7737d01f8, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.828077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.856694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:34.858850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75541e48-566c-4620-9f2f-a0bc151c2e3e service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:34.902530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 834/1670] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:34.974404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 837/1671] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:34 2022] PUT /identity/v3/projects/6f6ac69be54148afa3df243ca3b2bd9c/users/aa054f202b914d45ac67e8c7737d01f8/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:34.975079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:34.975393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:34.975672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:35.063944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] MFA Rules not processed for user `aa054f202b914d45ac67e8c7737d01f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:35.065719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:35.287940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 835/1672] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:36:35.290246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:35.412096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad1019-d759-43d9-9aa4-db3f45413e83 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:35.419167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:35.420076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:35.420408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:35.420734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:35.421445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:35.424186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:35.529886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 838/1673] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:35 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:35.535144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:35.583256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6050299-a6df-4d32-bf13-7e1fefd25dec None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:35.633876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:35.764697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:35.764697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:35.764697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:35.764697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:35.765135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:35.780662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:35.925131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 836/1674] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:35 2022] POST /identity/v3/users => generated 300 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:35.928901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:35.959530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6c229-e4ba-48fc-834d-ae6c52ec3f01 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:35.976302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:35.977891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:35.978376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:35.978857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:35.979835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:35.982526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:36.024760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 839/1675] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:36.026777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:36.049241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca081-3f2e-4cc0-bb33-cf67883c1340 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:36.078963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:36.080157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:36.080607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:36.081852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] PATH_INFO: `/v3/projects/3df15cf354df41e6975db8019ef33f53/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:36.105137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df15cf354df41e6975db8019ef33f53, user_id=c4ee4b9d5c8e4cdb9e7fbbf72ead73f6, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:36.108552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:36.214144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 837/1676] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:36 2022] PUT /identity/v3/projects/3df15cf354df41e6975db8019ef33f53/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:36.218061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:36.231671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80462a7-db94-41de-80a6-9d281f4850e9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:36.445715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:36.446905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:36.447379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:36.447862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:36.448811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:36.450710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c834e-5ac1-4a5d-8364-ebb33c9165c8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:36.454658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:36.471428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 840/1677] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:36.471789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:36.486315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f321b4-4d1e-4379-9363-a800b9aa6cfd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:36.502931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:36.504825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:36.507780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:36.509606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] PATH_INFO: `/v3/projects/3df15cf354df41e6975db8019ef33f53/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:36.512358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df15cf354df41e6975db8019ef33f53, user_id=c4ee4b9d5c8e4cdb9e7fbbf72ead73f6, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:36.517466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:36.566265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 838/1678] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:36 2022] PUT /identity/v3/projects/3df15cf354df41e6975db8019ef33f53/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:36.570338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:36.570633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:36.570914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:36.636497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] MFA Rules not processed for user `c4ee4b9d5c8e4cdb9e7fbbf72ead73f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:36.638106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:36.776944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 841/1679] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:36.799977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:36.805060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:37.035811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962659-064d-4eba-8b28-475a2b472ace admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:37.046893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:37.048173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:37.049959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:37.049959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:37.113121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:37.117885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba14b97-e3db-4a2b-93d4-7cbd0b062ccd service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:37.169053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 839/1680] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:37.425022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:37.589204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:37.589762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:37.589852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:37.623560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:37.630610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:37.634685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36971-a998-437a-8c74-2d0f40b0fb47 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:37.646115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:37.647281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:37.647701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:37.649421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] PATH_INFO: `/v3/users/aa054f202b914d45ac67e8c7737d01f8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:37.651369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa054f202b914d45ac67e8c7737d01f8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:37.657196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:37.879653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 840/1681] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:36:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:36:37.988059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:37.993231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.084404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa054f202b914d45ac67e8c7737d01f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.088645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa054f202b914d45ac67e8c7737d01f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.089993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a14d8-7bfd-4c75-b816-5430579890a1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:38.111431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:38.112360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:38.112644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:38.112915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] PATH_INFO: `/v3/users/3575512e61e24d14b802cd58542ffade` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:38.115598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3575512e61e24d14b802cd58542ffade)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:38.118063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:38.149826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa054f202b914d45ac67e8c7737d01f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.157014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa054f202b914d45ac67e8c7737d01f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.163439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:38.168126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-56dfe5f0-d593-4db4-83f1-bbe5ab4ebf97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa054f202b914d45ac67e8c7737d01f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.280813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 842/1682] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:37 2022] DELETE /identity/v3/users/aa054f202b914d45ac67e8c7737d01f8 => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:38.288300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:38.288589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:38.291953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:38.314525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:38.333976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:38.336001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.431003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 843/1683] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:38.436999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:38.438568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3575512e61e24d14b802cd58542ffade'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.441842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.467445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3575512e61e24d14b802cd58542ffade'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.525516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3575512e61e24d14b802cd58542ffade'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.527965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3575512e61e24d14b802cd58542ffade'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.655244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3575512e61e24d14b802cd58542ffade'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:38.668471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 841/1684] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:37 2022] DELETE /identity/v3/users/3575512e61e24d14b802cd58542ffade => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:38.671286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:38.674978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a050a59-ef5a-4be0-b78b-7ccb8833e470 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:38.676324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.837427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:38.839604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:38.839914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:38.840193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:38.844583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.875579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8214a-301e-4ad5-a942-c291a6f302c8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:38.881422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:38.882473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:38.883188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:38.883648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:38.886247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:38.933827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:38.936139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c63cd-1651-4d2e-b237-568d291db4ee service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:38.953959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 844/1685] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:36:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:36:38.972241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:38.975075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:38.975582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:39.076775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:39.085466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:39.087752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:39.095542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:39.106856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 842/1686] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:39.114523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 845/1687] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:39.172106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:39.175477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:39.203449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b8ab-3aa8-4708-b2ba-4a30ab366526 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:39.214644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:39.217273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:39.218226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:39.219674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:39.265110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:39.271152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12574af4-8394-4516-a024-25b8b7104a11 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:39.305674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 843/1688] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:36:39.919864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:40.166867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:40.168893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:40.204737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac317d3-2e1c-4683-bacb-738b6cbfb0d2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:40.220582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:40.221597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:40.221960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:40.222526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] PATH_INFO: `/v3/projects/1ae983f1ecde4623986967c0ef40c295` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:40.238137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae983f1ecde4623986967c0ef40c295)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:40.240280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:40.431407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae983f1ecde4623986967c0ef40c295', 'request_id': 'req-ae709718-f41b-44e4-955a-068238b9dade'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:40.448212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae983f1ecde4623986967c0ef40c295', 'request_id': 'req-ae709718-f41b-44e4-955a-068238b9dade'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:40.476695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae983f1ecde4623986967c0ef40c295', 'request_id': 'req-ae709718-f41b-44e4-955a-068238b9dade'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:40.614007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] The token cache is being invalidate because project 1ae983f1ecde4623986967c0ef40c295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:40.614331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:40.615014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae709718-f41b-44e4-955a-068238b9dade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:40.616623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 846/1689] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:40 2022] DELETE /identity/v3/projects/1ae983f1ecde4623986967c0ef40c295 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:40.621728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:40.622243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:40.622418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:40.644250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:40.645778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:40.671262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:40.743403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:40.743666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 844/1690] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:40.744500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:40.745032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:40.874434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:40.876723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:40.903727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 847/1691] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:40.907888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:40.918956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa609060-d721-4169-a0b9-e6716901b1cd None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:40.940646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:40.941645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:40.942074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:40.942520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:40.943501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:40.943959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:40.945973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a93128-aeac-4e68-8bdc-1e13ba7f4778 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:40.969121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 845/1692] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:40.969340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:40.991193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e4338-0f59-4ee2-b0c9-13644eb074e4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:40.999290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:41.000011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:41.000841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:41.001117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:41.001893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:41.010138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:41.053509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:41.054705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:41.056225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:41.130647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] MFA Rules not processed for user `e5081aed86004f6c99a05c9fbf2a36b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:41.132373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:41.169822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 848/1693] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:36:41.175757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:41.326660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c23b2-576e-49be-a742-f9ea02702714 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:41.326660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:41.326660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:41.326660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:41.327997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:41.328657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 846/1694] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:41.328846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:41.328846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:41.845078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:41.851144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:41.887422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 849/1695] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:41 2022] POST /identity/v3/users => generated 288 bytes in 697 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:41.887422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:41.910064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2d237-47dc-4d2d-91ed-756f64cda6fe admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:41.916183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:41.921220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:41.921645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:41.922563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:41.923589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:41.932160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:41.954051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 850/1696] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:41.955101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2b332-d0e0-4ee1-af93-e7a9ae99fbf7 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:41.955364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:41.979747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f1222-18ef-4367-9926-f2d3ad785de7 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:41.991286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:41.992169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:41.992456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:41.992727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] PATH_INFO: `/v3/projects/6f6ac69be54148afa3df243ca3b2bd9c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:41.999536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6ac69be54148afa3df243ca3b2bd9c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:42.001175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:42.002762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.003696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.004447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.004728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:42.007002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:42.010804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:42.050049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 851/1697] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:36:42.057453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:42.067998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:42.079731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed670-42d2-4461-afd3-a5d60a871838 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:42.091880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:42.095163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.096183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.096610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] PATH_INFO: `/v3/projects/9554d406b5f6411597602150f5c4377b/users/2f305fc5265249899699d26aad4ae633/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.184036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9554d406b5f6411597602150f5c4377b, user_id=2f305fc5265249899699d26aad4ae633, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:42.197614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a90bc1-6d0c-498b-9e15-fe68c55430a1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:42.287346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6ac69be54148afa3df243ca3b2bd9c', 'request_id': 'req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:42.295984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6ac69be54148afa3df243ca3b2bd9c', 'request_id': 'req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:42.304148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6ac69be54148afa3df243ca3b2bd9c', 'request_id': 'req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:42.353641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 852/1698] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:42 2022] PUT /identity/v3/projects/9554d406b5f6411597602150f5c4377b/users/2f305fc5265249899699d26aad4ae633/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:42.358305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6384bd-b88d-4142-8443-756f521f48d0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.360699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6384bd-b88d-4142-8443-756f521f48d0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.361175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6384bd-b88d-4142-8443-756f521f48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.395422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8d6384bd-b88d-4142-8443-756f521f48d0 None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:42.397599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8d6384bd-b88d-4142-8443-756f521f48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.399602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] The token cache is being invalidate because project 6f6ac69be54148afa3df243ca3b2bd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:42.399917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:42.400924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd60bac2-d7cf-4ac1-b1ca-54c58fb8cef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:42.402719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 847/1699] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:41 2022] DELETE /identity/v3/projects/6f6ac69be54148afa3df243ca3b2bd9c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:42.404164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.404449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.404723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.446389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] MFA Rules not processed for user `8d52fe5018e2450dad79596fb1129c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:42.447878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.514846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 853/1700] 10.222.0.16 () {58 vars in 1015 bytes} [Fri May 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:42.517281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828acb9-7807-432a-9940-c309dc631691 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.525007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828acb9-7807-432a-9940-c309dc631691 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.525552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828acb9-7807-432a-9940-c309dc631691 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.527658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 854/1701] 10.222.0.48 () {54 vars in 912 bytes} [Fri May 27 09:36:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:42.529296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707cf7-06f5-4aa6-a126-f525db7c9746 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.529758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707cf7-06f5-4aa6-a126-f525db7c9746 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.530176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707cf7-06f5-4aa6-a126-f525db7c9746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.556876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:42.579024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-52707cf7-06f5-4aa6-a126-f525db7c9746 None None] MFA Rules not processed for user `2f305fc5265249899699d26aad4ae633`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:42.591365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-52707cf7-06f5-4aa6-a126-f525db7c9746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.595931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 848/1702] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:42.603098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:42.603098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.779458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01de5e-a96c-4e6b-ba46-cd47e11a56f3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:42.793602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 855/1703] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:42.793814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:42.793814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.793814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.793814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] PATH_INFO: `/v3/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.794170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.794820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.795323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.795863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ee4b9d5c8e4cdb9e7fbbf72ead73f6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:42.800653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:42.848878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:42.851130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.875399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 856/1704] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:42.888036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:42.890088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:42.903695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe006bac-9e7b-44a7-a862-7a373ac27fed None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:42.915846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:42.917105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:42.917647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:42.918182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:42.931121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:42.935303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:42.949219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 857/1705] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:43.095387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ee4b9d5c8e4cdb9e7fbbf72ead73f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:43.111654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ee4b9d5c8e4cdb9e7fbbf72ead73f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:43.139801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ee4b9d5c8e4cdb9e7fbbf72ead73f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:43.142150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ee4b9d5c8e4cdb9e7fbbf72ead73f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:43.271373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ee4b9d5c8e4cdb9e7fbbf72ead73f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:43.296986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 849/1706] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:42 2022] DELETE /identity/v3/users/c4ee4b9d5c8e4cdb9e7fbbf72ead73f6 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:43.797551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:43.800169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:43.911554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:43.913480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:44.152094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:44.205882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc433-bb3d-4809-a9c1-b297fbdcf766 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:44.233237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:44.234478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:44.234935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:44.235385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:44.240997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:44.308880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fb2ee-fc48-4749-8703-22bb1f5ea2a6 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:44.315022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:44.316573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:44.380511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 850/1707] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:44.380511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:44.380511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:44.415675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:44.478174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:44.478174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:44.478174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:44.478174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:44.478835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:44.557533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:44.559388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:44.626758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 858/1708] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 831 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:44.742389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb85b-c9b5-4d4c-91e2-8329b586a034 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:44.753730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:44.754638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:44.755173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:44.755435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] PATH_INFO: `/v3/projects/3df15cf354df41e6975db8019ef33f53` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:44.757431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df15cf354df41e6975db8019ef33f53)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:44.759563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:44.959361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df15cf354df41e6975db8019ef33f53', 'request_id': 'req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:44.980973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df15cf354df41e6975db8019ef33f53', 'request_id': 'req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:44.991580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df15cf354df41e6975db8019ef33f53', 'request_id': 'req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.024596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:45.028165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:45.078233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] The token cache is being invalidate because project 3df15cf354df41e6975db8019ef33f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:45.079726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.080956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a4b97f38-b05f-4cf7-9ae2-990fa7b5b91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.087137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 851/1709] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:44 2022] DELETE /identity/v3/projects/3df15cf354df41e6975db8019ef33f53 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:45.108088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:45.108088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:45.108088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:45.108465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f18a6-741d-410f-823c-0ee24c0ae55f service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:45.152685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:45.152685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:45.276055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:45.277789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:45.278099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:45.278461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] PATH_INFO: `/v3/users/32ccc18481774bdcaf28dedf0edbd56c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:45.280548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ccc18481774bdcaf28dedf0edbd56c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:45.283196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:45.344300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 852/1710] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:45.345585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:45.358561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:45.391529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37e7cd-e56d-4aa3-b9dc-07e97468b1a6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:45.410232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:45.411231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:45.411754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:45.412247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] PATH_INFO: `/v3/users/2f305fc5265249899699d26aad4ae633` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:45.414365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f305fc5265249899699d26aad4ae633)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:45.417716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:45.624053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:45.663341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ccc18481774bdcaf28dedf0edbd56c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.664560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ccc18481774bdcaf28dedf0edbd56c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.725187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ccc18481774bdcaf28dedf0edbd56c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.726091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ccc18481774bdcaf28dedf0edbd56c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.738560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-b8c57a44-e58f-4c3b-905b-d6a70774610a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ccc18481774bdcaf28dedf0edbd56c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.773068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:45.862678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 859/1711] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:45 2022] DELETE /identity/v3/users/32ccc18481774bdcaf28dedf0edbd56c => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:45.866436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff66aa-4723-4ac2-9176-04d6f399757f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:45.868639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff66aa-4723-4ac2-9176-04d6f399757f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:45.869078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff66aa-4723-4ac2-9176-04d6f399757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:45.871289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f305fc5265249899699d26aad4ae633'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.907226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f305fc5265249899699d26aad4ae633'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.940252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5aff66aa-4723-4ac2-9176-04d6f399757f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:45.950288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5aff66aa-4723-4ac2-9176-04d6f399757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:45.972818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f305fc5265249899699d26aad4ae633'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:45.980448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f305fc5265249899699d26aad4ae633'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:46.100365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-95cd2587-454b-4ab6-a50a-c25bc377d73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f305fc5265249899699d26aad4ae633'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:46.130103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 853/1712] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:45 2022] DELETE /identity/v3/users/2f305fc5265249899699d26aad4ae633 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:46.138424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.139466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.140428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.167225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 860/1713] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:46.173044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.173882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.174648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.213906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:46.216346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:46.237000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:46.242854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:46.378094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 861/1714] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:46.379059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:46.404286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e9a8c-23d6-4515-9069-a49f4a7048d8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:46.411725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:46.413059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.413505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.416087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.433365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:46.435335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:46.440320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:46.480490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 862/1715] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:46.481677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:46.483610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:46.542594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 854/1716] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:46.545401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:46.565424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20546edd-5748-4d54-a733-715a2ba7c6a8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:46.573293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:46.574948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.575741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.576489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.578687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:46.583773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:46.631108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:46.654246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6f4ba-fe73-4a14-a165-07ccb9c68c1e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:46.662234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:46.663701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.664309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.665588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.688428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:46.688428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd429d10-5823-49c3-b3a6-06b64a5c337b service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:46.728932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 863/1717] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:46.730922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 855/1718] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:46 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:46.735609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:46.747800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf102-c82d-4da8-8b04-af2788d80e3f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.756511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf102-c82d-4da8-8b04-af2788d80e3f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.758373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedf102-c82d-4da8-8b04-af2788d80e3f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:46.763276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 864/1719] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:36:46 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:46.764244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedf102-c82d-4da8-8b04-af2788d80e3f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:46.924994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69193-d3d1-4432-aff4-adc82758b2b6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:46.943424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:46.943424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:46.943424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:46.948416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.037088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedf102-c82d-4da8-8b04-af2788d80e3f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:47.043732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:47.044764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.044959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.045436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.047965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.053358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:47.153327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.155519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3a31d-f572-49f8-96fb-1481f10df8bf service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:47.197392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:47.204356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 856/1720] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:47.206389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 865/1721] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:46 2022] POST /identity/v3/users => generated 291 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:47.212978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:47.221510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc98b5-1f52-4453-8552-bfa41b2dcc33 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.222243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc98b5-1f52-4453-8552-bfa41b2dcc33 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.222804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc98b5-1f52-4453-8552-bfa41b2dcc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.235061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014f47-0a24-401a-847d-d3ceffcdcb43 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:47.251090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:47.251895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.252180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.252471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.256316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.260690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:47.287931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 866/1722] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:47.290491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:47.301836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d6bc98b5-1f52-4453-8552-bfa41b2dcc33 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:47.304392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d6bc98b5-1f52-4453-8552-bfa41b2dcc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:47.322245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44055e9-67d4-429c-96a9-de6dcffcbf7b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:47.362693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:47.363404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.363683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.363980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.364737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.366416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:47.408828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 867/1723] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:47.410041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:47.449771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a0156-6710-4ab5-b941-3c87c4a8c74b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:47.464592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:47.466420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.467045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.467789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] PATH_INFO: `/v3/projects/060a66502d93405c8b2aa01d16ac7d8d/users/d6d48c17e3a34620b1fb8fd6640be80e/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.536483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=060a66502d93405c8b2aa01d16ac7d8d, user_id=d6d48c17e3a34620b1fb8fd6640be80e, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.541764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:47.649734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 868/1724] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:47 2022] PUT /identity/v3/projects/060a66502d93405c8b2aa01d16ac7d8d/users/d6d48c17e3a34620b1fb8fd6640be80e/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:47.651083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:47.663555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:47.716014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 857/1725] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:47.718395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.718905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.719453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.875969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddbac3-016a-480f-abda-8ef960642fae admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:47.889377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] MFA Rules not processed for user `d6d48c17e3a34620b1fb8fd6640be80e`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:47.891660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:47.901883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:47.904834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:47.908467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:47.909005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] PATH_INFO: `/v3/users/66bc9b7cbe98440ea5cc5270ae6973ec` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:47.915014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66bc9b7cbe98440ea5cc5270ae6973ec)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:47.917474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:48.070437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:48.146623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 858/1726] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:48.146623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:48.151794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:48.201472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66bc9b7cbe98440ea5cc5270ae6973ec'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.202177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66bc9b7cbe98440ea5cc5270ae6973ec'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.246520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66bc9b7cbe98440ea5cc5270ae6973ec'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.248673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66bc9b7cbe98440ea5cc5270ae6973ec'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.266271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66bc9b7cbe98440ea5cc5270ae6973ec'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.297796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:48.326162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92acb8-b020-4530-84a4-52a33d88f70b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:48.332790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:48.333472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:48.333776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:48.334231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:48.336622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:48.384437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 869/1727] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:47 2022] DELETE /identity/v3/users/66bc9b7cbe98440ea5cc5270ae6973ec => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:48.386694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:48.389910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:48.392891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:48.404398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:48.451045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 859/1728] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:48.455221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:48.460719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:48.491280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20592421-6d40-4a81-a468-ff2db93468e4 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:48.636325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95cdf-251e-49aa-bfc6-327504b90500 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:48.656175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:48.660386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:48.660817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:48.662256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] PATH_INFO: `/v3/projects/9554d406b5f6411597602150f5c4377b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:48.667010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9554d406b5f6411597602150f5c4377b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:48.674241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:48.699637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:48.703272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:48.703651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:48.708907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:48.708907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:48.708907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e2212-fa30-4754-bd9a-f3a0c864a69f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:48.861049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9554d406b5f6411597602150f5c4377b', 'request_id': 'req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.871498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9554d406b5f6411597602150f5c4377b', 'request_id': 'req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.881303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9554d406b5f6411597602150f5c4377b', 'request_id': 'req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:48.908070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 860/1729] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:48 2022] POST /identity/v3/projects => generated 381 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:48.909519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3698fa-5d24-437a-a431-4f85432bbc5f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:48.916141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3698fa-5d24-437a-a431-4f85432bbc5f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:48.916551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3698fa-5d24-437a-a431-4f85432bbc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:48.974475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6b3698fa-5d24-437a-a431-4f85432bbc5f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:48.977972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6b3698fa-5d24-437a-a431-4f85432bbc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.010689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] The token cache is being invalidate because project 9554d406b5f6411597602150f5c4377b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:49.011166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.011870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.023611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 870/1730] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:48 2022] DELETE /identity/v3/projects/9554d406b5f6411597602150f5c4377b => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:49.023611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.027994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.117993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 861/1731] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:49.119724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.120073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.120354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.145610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:49.147136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.176995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 862/1732] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:49.176995 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.192351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.221505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:49.261027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ee9a-ef09-46e7-afa5-e98dd9d7de79 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.268877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.269639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.270243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.270779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.274397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.278993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.375866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3544c8-efc8-48b2-891e-413ccdc5db2d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.384376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.385746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.386268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.387328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.397738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 871/1733] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:49 2022] POST /identity/v3/users => generated 297 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:49.398951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.400923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.427665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac2c40-8eda-43ab-834b-f28f5188bd3e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.432338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.436081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd61cd-5666-48f6-914f-278d186ad7eb service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.448018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.448921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.449202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.449480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] PATH_INFO: `/v3/projects/1598c6c82b11451fa6851486e117d6f4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.455453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1598c6c82b11451fa6851486e117d6f4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.457597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.459009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 863/1734] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:49.460404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.460691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.461108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.479703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:49.496142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:49.521931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.551250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 864/1735] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:49.552542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.571743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db951360-2b17-4348-888c-446362d5cdf1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.588430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.589866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.590447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.591004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.592433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.597403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.630967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 865/1736] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:49.635139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.664152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1598c6c82b11451fa6851486e117d6f4', 'request_id': 'req-80a4ae5f-cf80-4596-aa79-138b1a8817b6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.669244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e183a-cbfe-40d5-b437-c03bb934f830 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.682004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1598c6c82b11451fa6851486e117d6f4', 'request_id': 'req-80a4ae5f-cf80-4596-aa79-138b1a8817b6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.701397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1598c6c82b11451fa6851486e117d6f4', 'request_id': 'req-80a4ae5f-cf80-4596-aa79-138b1a8817b6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.770401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.772249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.772660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.773069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.779331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:49.779331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.782688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.812790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] The token cache is being invalidate because project 1598c6c82b11451fa6851486e117d6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:49.813262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.815360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:49.817913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 866/1737] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:49.819458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 872/1738] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:49 2022] DELETE /identity/v3/projects/1598c6c82b11451fa6851486e117d6f4 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:49.819750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.821069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:49.821879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.824132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:49.840630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedc54b-274c-42bf-b624-e3d421dd3773 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.848554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.850082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.850425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.850709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] PATH_INFO: `/v3/projects/9f23c513faf24335bf5095c78dfb9902/users/1cfb96389c454624b6749856a5417afd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.853898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4ae5f-cf80-4596-aa79-138b1a8817b6 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:49.872110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f23c513faf24335bf5095c78dfb9902, user_id=1cfb96389c454624b6749856a5417afd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.875712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb143b-d6e1-48a8-9254-75bf4024d748 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.885664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:49.886904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.887381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.887728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:49.889622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:49.892812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:49.978614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 867/1739] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:49 2022] PUT /identity/v3/projects/9f23c513faf24335bf5095c78dfb9902/users/1cfb96389c454624b6749856a5417afd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:49.981406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:49.981888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:49.983155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.035124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 873/1740] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:49 2022] POST /identity/v3/projects => generated 403 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:50.035871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:50.037687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:50.038437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:50.144921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 868/1741] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:50.146104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:50.179095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc0acf-1257-4c6e-80c8-74f1bd17b318 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:50.197567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:50.198444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.198732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.199141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.205176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:50.208772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:50.312986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e1c9-37b5-43ae-b6ac-5d7d490e623c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:50.322158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:50.323557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.324829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.324829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.326470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:50.330595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85876cc-4dbd-47ba-a5d1-35c84ff5b4ee admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:50.355540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 874/1742] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:50.363277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.363841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.366228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.433827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 869/1743] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:50 2022] POST /identity/v3/users => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:50.433827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:50.478450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:50.480658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:50.499438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9fad-def9-460e-ae19-24991a8d4521 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:50.516733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:50.518461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.518978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.538277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] PATH_INFO: `/v3/projects/9f23c513faf24335bf5095c78dfb9902/users/1cfb96389c454624b6749856a5417afd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.545844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 875/1744] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:50.545844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:50.549855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f23c513faf24335bf5095c78dfb9902, user_id=1cfb96389c454624b6749856a5417afd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:50.557435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:50.578974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4df7ec-52e4-4d5d-9e2a-f0fcccc57bdc None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:50.590608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 870/1745] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:50 2022] PUT /identity/v3/projects/9f23c513faf24335bf5095c78dfb9902/users/1cfb96389c454624b6749856a5417afd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:50.592254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:50.615654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:50.718337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:50.757061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:50.758407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.760043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.760502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.762686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:50.764771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6865d9-4e52-4f5c-844b-9465d3f469c8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:50.824781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 876/1746] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:50.841713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.846492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.849025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.907234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] MFA Rules not processed for user `1cfb96389c454624b6749856a5417afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:50.917577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:50.924920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3f76-b5c0-49ba-a44b-34a008cc7c2e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:50.945759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:50.949638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:50.952341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:50.958817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:50.961812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:50.962132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:50.976219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:51.034476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 871/1747] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:51.036212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:51.041250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:51.062981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c3ff3-61bc-46f6-bf48-c51198af88ca admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:51.091617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331340de-987a-42b5-b175-2279927669cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:51.096566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331340de-987a-42b5-b175-2279927669cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:51.098762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331340de-987a-42b5-b175-2279927669cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:51.101262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331340de-987a-42b5-b175-2279927669cc admin admin] PATH_INFO: `/v3/projects/e10012e947154b95b38fe47f16c88ab6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:51.102871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331340de-987a-42b5-b175-2279927669cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e10012e947154b95b38fe47f16c88ab6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:51.107572 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331340de-987a-42b5-b175-2279927669cc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:51.188007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 877/1748] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:51.189249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:51.222817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100717e8-b96b-4af3-9af5-b8a490f52a9f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:51.251713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:51.252635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:51.252976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:51.253291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:51.256596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:51.257964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:51.275515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 878/1749] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:51.276609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:51.279051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:51.295657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae966c-9af3-4341-919d-0be7af581bef admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:51.302042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:51.303104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:51.303420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:51.305177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] PATH_INFO: `/v3/users/2ad1ad3922d4431391c938db27cbc280` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:51.306689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad1ad3922d4431391c938db27cbc280)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:51.311563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:51.383822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e10012e947154b95b38fe47f16c88ab6', 'request_id': 'req-331340de-987a-42b5-b175-2279927669cc'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.397418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e10012e947154b95b38fe47f16c88ab6', 'request_id': 'req-331340de-987a-42b5-b175-2279927669cc'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.426019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e10012e947154b95b38fe47f16c88ab6', 'request_id': 'req-331340de-987a-42b5-b175-2279927669cc'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.597259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] The token cache is being invalidate because project e10012e947154b95b38fe47f16c88ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:51.598157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.600376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.607946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 872/1750] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:51 2022] DELETE /identity/v3/projects/e10012e947154b95b38fe47f16c88ab6 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:51.607946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:51.610130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:51.695938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad1ad3922d4431391c938db27cbc280'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.697382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad1ad3922d4431391c938db27cbc280'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.730176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad1ad3922d4431391c938db27cbc280'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.731580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad1ad3922d4431391c938db27cbc280'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.747730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad1ad3922d4431391c938db27cbc280'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:51.808544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331340de-987a-42b5-b175-2279927669cc admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:51.816876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:51.818438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:51.818949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:51.819455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:51.820951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:51.825163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:51.875799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:51.897746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 879/1751] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:51 2022] DELETE /identity/v3/users/2ad1ad3922d4431391c938db27cbc280 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:51.898816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:51.901241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.022298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 873/1752] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:51 2022] POST /identity/v3/projects => generated 363 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:52.027524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:52.034829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.058697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6c7fb-80a4-4b35-842c-60bb0f1b711c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:52.207569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:52.214901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:52.216043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.217122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.217331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.221804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.311322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be3ee-e576-4613-b208-ea5d39d7c29e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:52.319748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:52.320771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.323184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.323184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] PATH_INFO: `/v3/projects/f72038fab1624d6285967d66f2e8098c/users/d241bafb09094103885a4389a4b70050/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.352215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f72038fab1624d6285967d66f2e8098c, user_id=d241bafb09094103885a4389a4b70050, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:52.355653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bab32-d4b8-4a68-a82b-a0748bfad0f9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:52.429726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 874/1753] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:52 2022] PUT /identity/v3/projects/f72038fab1624d6285967d66f2e8098c/users/d241bafb09094103885a4389a4b70050/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:52.433832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.434171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.438595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.466146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:52.469898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454c5b9-291b-4195-9bd8-95fd521ef37b service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:52.483212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:52.485133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.486526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 880/1754] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:52.487669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e574e2-227c-40b0-ba33-620163c93bc3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.488237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e574e2-227c-40b0-ba33-620163c93bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.488426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e574e2-227c-40b0-ba33-620163c93bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.565539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-78e574e2-227c-40b0-ba33-620163c93bc3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:52.574499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-78e574e2-227c-40b0-ba33-620163c93bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.768811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 875/1755] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:52.769899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:52.771768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.813442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd87cf9-194e-4d8f-b289-5c49a5334abc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:52.814409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 881/1756] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:52.818156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c008-8fd1-40f9-9f5a-1dd00883a132 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.818966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c008-8fd1-40f9-9f5a-1dd00883a132 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.819473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101c008-8fd1-40f9-9f5a-1dd00883a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.826590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:52.827304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.827739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.828602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:52.829363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:52.832539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:52.837615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:52.864054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d101c008-8fd1-40f9-9f5a-1dd00883a132 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:52.865897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d101c008-8fd1-40f9-9f5a-1dd00883a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:52.884842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 882/1757] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:52.887152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:52.887711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:52.888170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.039263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:53.042699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:53.077804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 883/1758] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:53.077804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.082680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:53.103086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 876/1759] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:52 2022] POST /identity/v3/users => generated 282 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:53.108081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.113104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:53.206920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c383904-4d34-4c8f-a7b2-343ca4d6ba6a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.217329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.218456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.218958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.219436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.251548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.256334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.287755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d1ac-6ed0-48ab-963e-39b0136ec44b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.289958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 884/1760] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:53.297948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.299839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.300581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.300882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.301150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.335511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.336977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.341752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e4e8-daa9-49d2-936f-1954e433e812 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.363115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.364302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.365377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.366780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.368087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 877/1761] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:53.368282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:53.368740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.369044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.382296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.404972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f9cc8-96c7-4e82-a3e5-d0cc9fe0dfba service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.426001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.426784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.427053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.428479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.429384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.441102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.467248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 885/1762] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:53.471393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.480563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 878/1763] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:53.484034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.506811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2a4eb-6288-44d8-be64-527e27481027 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.507851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418197df-3f62-4a35-bda3-f92d6b3ede8b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.514270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.528522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.528869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.529048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.532506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.534282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.536653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.537785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.538376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.538653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.540014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.543318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.581116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 879/1764] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:53.585441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.610793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db99aef-a0dd-47bc-97ca-9c16ddafbe90 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:53.627292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:53.631904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.632242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.632586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] PATH_INFO: `/v3/projects/aaf2855f72284b18bc517517722493ca/users/1648324e8b134098a0365bcde058e289/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.698520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf2855f72284b18bc517517722493ca, user_id=1648324e8b134098a0365bcde058e289, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:53.704087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc4e98-9030-4c9a-9379-b097c2a88b92 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:53.708572 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:53.762180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 886/1765] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:53 2022] POST /identity/v3/projects => generated 371 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:53.766299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:53.833950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 880/1766] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:53 2022] PUT /identity/v3/projects/aaf2855f72284b18bc517517722493ca/users/1648324e8b134098a0365bcde058e289/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:53.857945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:53.858535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:53.859660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:53.998233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] MFA Rules not processed for user `1648324e8b134098a0365bcde058e289`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:54.001003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:54.217646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e8f99-0621-4236-97a6-1074c82e9127 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:54.308621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 881/1767] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:54.315805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:54.536563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:54.538432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:54.538970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:54.542634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:54.544182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:54.559429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:54.605890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:54.629014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a586ec-099a-4293-b0c1-59c8a9d7f1d8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:54.646660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:54.648600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:54.649645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:54.650382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:54.652278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:54.676798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:54.719814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 887/1768] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:53 2022] POST /identity/v3/users => generated 286 bytes in 954 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:36:54.721503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:54.900981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 882/1769] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:54 2022] POST /identity/v3/projects => generated 361 bytes in 587 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:36:54.902331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:54.939213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844408-1d38-48e5-8a2b-ede3af78526f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.129081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b914ac-30ca-49c8-b1eb-978e2d3dd8f5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.136771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cfb96389c454624b6749856a5417afd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f23c513faf24335bf5095c78dfb9902', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.138502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.139001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.139409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.141333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:55.141773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.144434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.161342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 888/1770] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:55.165117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.172324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.173536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.175969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.176886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.178432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.182306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.183477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe12d4c-5362-4fc2-a265-e8c6002f4c61 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.191500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.192299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.192640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.192938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.193856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.196011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.223234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 883/1771] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:55.224012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.234227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:55.242779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a505fa-67db-49e8-9621-8df3db550e32 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.255009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cfb96389c454624b6749856a5417afd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f23c513faf24335bf5095c78dfb9902', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.256390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.256948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.258030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.258948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.263452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.352977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 889/1772] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:55 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:55.366376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.380089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05696f-c533-40b9-be83-9aeb07c25b6b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.446554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.446554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.446554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.446554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.464258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.464258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.464258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 890/1773] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:55.464258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.478993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785feea-163c-4a85-adef-fd49365c2f57 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.506085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 884/1774] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:55 2022] POST /identity/v3/projects => generated 354 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:55.530521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.530837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.530925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.530925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.530925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.530925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.530925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.642140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 891/1775] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:55.642140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.670640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b872-9d3c-4a42-902e-9b0d51343c60 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.725781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b76ea-3ef8-4371-9c59-618bc626ed1e tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:55.760719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.761697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.762030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.762353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] PATH_INFO: `/v3/projects/dcf66f7181f34c6baa3fe60a124d6a12/users/5ec4a03199344617bb80c589de6d0273/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.790618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf66f7181f34c6baa3fe60a124d6a12, user_id=5ec4a03199344617bb80c589de6d0273, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.796400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.888411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:55.899037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cfb96389c454624b6749856a5417afd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f23c513faf24335bf5095c78dfb9902', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:55.899037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:55.899037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:55.899769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:55.900754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:36:55.901279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:55.904064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:55.938488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 892/1776] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:36:55.943244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:55.978395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31230b-aca6-4a1b-a206-0b57ce39cbc3 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:56.033111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 885/1777] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:55 2022] PUT /identity/v3/projects/dcf66f7181f34c6baa3fe60a124d6a12/users/5ec4a03199344617bb80c589de6d0273/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:56.042297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:56.063129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:56.216851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:56.216851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:56.216851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:56.224397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] PATH_INFO: `/v3/projects/6eb199bc06c84188b9fd09140b97a33f/users/dfcd2a135d9640e29535bdfbdf4a1cb9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:56.278279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb199bc06c84188b9fd09140b97a33f, user_id=dfcd2a135d9640e29535bdfbdf4a1cb9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:56.290957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad829f61-ae88-4123-805a-31b06c2b3d1e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:56.334965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534d375-f315-474c-93f0-e7b25bc710d9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:56.342089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:56.343920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:56.344273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:56.344554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] PATH_INFO: `/v3/projects/9015a8cf2a9040fba7060e4d377d99ed` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:56.346959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9015a8cf2a9040fba7060e4d377d99ed)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:56.348966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:56.400180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 893/1778] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:55 2022] PUT /identity/v3/projects/6eb199bc06c84188b9fd09140b97a33f/users/dfcd2a135d9640e29535bdfbdf4a1cb9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:56.400180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:56.401195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:56.401553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:56.451506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] MFA Rules not processed for user `5ec4a03199344617bb80c589de6d0273`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:56.453585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:56.522463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9015a8cf2a9040fba7060e4d377d99ed', 'request_id': 'req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:56.535369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9015a8cf2a9040fba7060e4d377d99ed', 'request_id': 'req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:56.554707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9015a8cf2a9040fba7060e4d377d99ed', 'request_id': 'req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:56.560408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:56.568110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 894/1779] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:56.568110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:56.692955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] The token cache is being invalidate because project 9015a8cf2a9040fba7060e4d377d99ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:56.693184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:56.694058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:56.700700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 886/1780] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:56 2022] DELETE /identity/v3/projects/9015a8cf2a9040fba7060e4d377d99ed => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:56.702295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:56.712824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:56.780295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4b47-41d7-4eb6-b521-3c70331d70d1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:56.819846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:56.885261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c61d58-f5b5-4308-bb7e-03a5b37dd4ab admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:56.892729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:56.896116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:56.896116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:56.896116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:56.896116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:56.898069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:56.935961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 887/1781] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:56.937696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:56.940062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:56.957387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dea7a-0ac7-4943-9722-5a4e811cc645 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:56.964443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:56.965838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:56.966397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:56.967068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:56.968299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:56.971272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:57.002265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cfb96389c454624b6749856a5417afd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f23c513faf24335bf5095c78dfb9902', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:57.008020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.008020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.008020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] PATH_INFO: `/v3/projects/7b2f9adf058549099e23c75f485b7513` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.008670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7b2f9adf058549099e23c75f485b7513)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:57.011169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:57.160013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 888/1782] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:56 2022] POST /identity/v3/projects => generated 373 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:57.161745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:57.188272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:57.197437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2f9adf058549099e23c75f485b7513', 'request_id': 'req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.209389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2f9adf058549099e23c75f485b7513', 'request_id': 'req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.218331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2f9adf058549099e23c75f485b7513', 'request_id': 'req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.308664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] The token cache is being invalidate because project 7b2f9adf058549099e23c75f485b7513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:57.309153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.310050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d3eb987c-e9fa-444f-9570-b8a5dbbf6f33 tempest-VolumeQuotaClassesTest-2043078529 tempest-VolumeQuotaClassesTest-2043078529-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.311865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 895/1783] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:56 2022] DELETE /identity/v3/projects/7b2f9adf058549099e23c75f485b7513 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:57.313940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.314236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.314442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.316419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 896/1784] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:36:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:36:57.321244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:57.324157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:57.339687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789440-142e-49c2-85bc-0008a7a603bc admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:57.447482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398a28d-76a9-4446-90f7-67597294de11 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:57.452573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:57.453532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.453875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:57.454105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.454397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] PATH_INFO: `/v3/projects/6eb199bc06c84188b9fd09140b97a33f/users/dfcd2a135d9640e29535bdfbdf4a1cb9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.454999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.455456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.455900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] PATH_INFO: `/v3/users/dad9e02337e340f3a244ef78f0f89e31` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.456785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb199bc06c84188b9fd09140b97a33f, user_id=dfcd2a135d9640e29535bdfbdf4a1cb9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:57.457944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad9e02337e340f3a244ef78f0f89e31)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:57.460274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:57.477989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:57.539173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 897/1785] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:57 2022] PUT /identity/v3/projects/6eb199bc06c84188b9fd09140b97a33f/users/dfcd2a135d9640e29535bdfbdf4a1cb9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:57.540671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:57.549791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:57.660469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d12f3c-9c2d-44c6-ac24-6eaf92b7ec78 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:57.673966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:57.674770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.675131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.675416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.676163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:57.678349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9eb38-c79c-419c-9e99-831303a8661d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:57.715321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad9e02337e340f3a244ef78f0f89e31'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.736046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad9e02337e340f3a244ef78f0f89e31'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.762793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad9e02337e340f3a244ef78f0f89e31'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.764108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad9e02337e340f3a244ef78f0f89e31'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.822754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 898/1786] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:57 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:57.825589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.826153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.826921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.879850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:57.882351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:57.905065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-0abb511b-b159-4f0a-98d2-a4f32ff52b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad9e02337e340f3a244ef78f0f89e31'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:57.923018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 889/1787] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:57 2022] DELETE /identity/v3/users/dad9e02337e340f3a244ef78f0f89e31 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:57.923460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:57.923965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:57.925271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:57.995091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] MFA Rules not processed for user `dfcd2a135d9640e29535bdfbdf4a1cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:57.997414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.058703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 899/1788] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:36:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:58.060895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.063894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.101459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b770af-3f1d-49d4-8b77-5820f6cf4f84 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.169181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 890/1789] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:57 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:58.169181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.182652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.207405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.208418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.208920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.209215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.210510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.212220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.225366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 900/1790] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:36:58.226489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.230864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.318636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749141-4317-4958-b4c1-2bc2e5ab9cd8 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.324013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.324774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.325060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.325374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.339838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.341413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.355534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 901/1791] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:58 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:58.356421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.375456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf0233-bf18-4a1e-98e3-d2029a4f1631 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.376837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530cbda-8c12-4363-8288-2d4976e1dd52 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.377366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.378379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.378664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.378940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] PATH_INFO: `/v3/projects/506a399779b2447fa1d3a2b70ad73880/users/adcefcef3036414a8bcc93bca5c7c84a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.401433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506a399779b2447fa1d3a2b70ad73880, user_id=adcefcef3036414a8bcc93bca5c7c84a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.406123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.407323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.408666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.409079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.409581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.426586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.429088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.442659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 891/1792] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:58.445859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.447913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.472744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe820c-9b88-4fa8-b7f8-a484d546942f service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.481899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.484729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.485226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.485911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] PATH_INFO: `/v3/users/d6d48c17e3a34620b1fb8fd6640be80e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.488891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d48c17e3a34620b1fb8fd6640be80e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.492471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.578520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 902/1793] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:58 2022] PUT /identity/v3/projects/506a399779b2447fa1d3a2b70ad73880/users/adcefcef3036414a8bcc93bca5c7c84a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:58.584061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.735793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eb063-d372-43ef-8692-76e868917796 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.741556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.742320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.742612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.742889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.743643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.746409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.772155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 903/1794] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:36:58.779575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:58.784908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:58.806132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0e8c-05e7-401f-aff4-2d9752c2fa4c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:58.814258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:58.815857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.815857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.816191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] PATH_INFO: `/v3/projects/506a399779b2447fa1d3a2b70ad73880/users/adcefcef3036414a8bcc93bca5c7c84a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.820991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506a399779b2447fa1d3a2b70ad73880, user_id=adcefcef3036414a8bcc93bca5c7c84a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:58.830339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb638a-d92e-42bc-94a2-57423f36fc11 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:58.832771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d48c17e3a34620b1fb8fd6640be80e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:58.842845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d48c17e3a34620b1fb8fd6640be80e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:58.853763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 904/1795] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:36:58 2022] PUT /identity/v3/projects/506a399779b2447fa1d3a2b70ad73880/users/adcefcef3036414a8bcc93bca5c7c84a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:58.855624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914184-3242-4cdb-8075-b603abaf7218 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.856679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914184-3242-4cdb-8075-b603abaf7218 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.857029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914184-3242-4cdb-8075-b603abaf7218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:58.882870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d48c17e3a34620b1fb8fd6640be80e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:58.882870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d48c17e3a34620b1fb8fd6640be80e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:58.898169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-30914184-3242-4cdb-8075-b603abaf7218 None None] MFA Rules not processed for user `adcefcef3036414a8bcc93bca5c7c84a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:58.899964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-30914184-3242-4cdb-8075-b603abaf7218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:58.948131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4cebfe57-7e93-452b-9058-302a81bc8938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d48c17e3a34620b1fb8fd6640be80e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:58.966237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 892/1796] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:36:58 2022] DELETE /identity/v3/users/d6d48c17e3a34620b1fb8fd6640be80e => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:58.968850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:58.969137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:58.971941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.044162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] MFA Rules not processed for user `d241bafb09094103885a4389a4b70050`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:59.048277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.070277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 905/1797] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:59.072442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.072901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.073307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.101872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:36:59.103350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.160007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 893/1798] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:59.160907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.166253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.226040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 906/1799] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:36:59.227362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.229553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.246937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c6368-2eb6-471f-b3c6-20acb11dffff None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.252509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.253255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.253583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.253851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.254785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513d5c7-b71d-4699-8ec2-d166d9d4a260 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.254949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.260249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.269778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.270991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.271418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.271865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.349861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.351579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.363264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 894/1800] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:59.364381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.366385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.387409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5156-9ca6-422d-b81e-35cbd3132330 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.393696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.398755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.398755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.398755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] PATH_INFO: `/v3/projects/5bb8f316d0984031bfa9c0062d2a1672` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.398755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:36:59.398755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb8f316d0984031bfa9c0062d2a1672)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.401059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.441004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 907/1801] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:36:59 2022] POST /identity/v3/projects => generated 405 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:59.441004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.441004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.508708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adcc9-26e9-44cd-aebb-edb603106d72 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.512898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.513628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.513911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.514215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.587489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb8f316d0984031bfa9c0062d2a1672', 'request_id': 'req-5b203305-7076-49ef-a960-83b883053181'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:59.596739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb8f316d0984031bfa9c0062d2a1672', 'request_id': 'req-5b203305-7076-49ef-a960-83b883053181'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:59.605837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb8f316d0984031bfa9c0062d2a1672', 'request_id': 'req-5b203305-7076-49ef-a960-83b883053181'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:59.641474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.644680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.663954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 908/1802] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:59.664823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.678735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5385-73a5-4150-901a-d23242bd6b76 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.730354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] The token cache is being invalidate because project 5bb8f316d0984031bfa9c0062d2a1672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:36:59.730660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:59.731424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:36:59.733684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 895/1803] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:36:59 2022] DELETE /identity/v3/projects/5bb8f316d0984031bfa9c0062d2a1672 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:36:59.734239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.735795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.790544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.792905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.793555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.794240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.795504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.799304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.837374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b203305-7076-49ef-a960-83b883053181 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:36:59.843880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:36:59.847148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.847345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.847511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.852620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.939746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:36:59.941229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f917-ccd3-40e7-9332-2fd68f9007e4 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:36:59.953573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 896/1804] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:36:59.957899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 909/1805] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:36:59 2022] POST /identity/v3/users => generated 309 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:36:59.960088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:36:59.960715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:36:59.961085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:36:59.965266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:36:59.970700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:36:59.998109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:00.001143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:00.021813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 897/1806] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:00.021813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.021813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:00.041728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f06e0-d22b-4ae5-a161-75c74398c6d3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.041996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323737b-1b4e-424c-a9b4-61e9c9fe8487 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.051431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.053060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.053449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.054119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] PATH_INFO: `/v3/users/1648324e8b134098a0365bcde058e289` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.054498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.055852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.056298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.056657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1648324e8b134098a0365bcde058e289)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.056774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.057963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.063342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3658e3-9232-427d-b860-d40eb250709b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.063553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.107882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 910/1807] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:00.107882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.107882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.107882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.138529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:00.140176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:00.149071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 911/1808] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:00.149909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.160317 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471deb3-e06f-43b4-98cf-d4455ccd65bd None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.166966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.170150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.170150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.170150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] PATH_INFO: `/v3/projects/509825b6bcb24e5dbc39bcf09987302a/users/1f647a1e051c496ba15e64885f0558de/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.192893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509825b6bcb24e5dbc39bcf09987302a, user_id=1f647a1e051c496ba15e64885f0558de, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.195993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.313651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 912/1809] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:00 2022] PUT /identity/v3/projects/509825b6bcb24e5dbc39bcf09987302a/users/1f647a1e051c496ba15e64885f0558de/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:00.314816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.395867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1648324e8b134098a0365bcde058e289'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:00.414620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b26c1-ca5d-42d1-b534-b4aa296da1d5 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.424485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.425902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.426450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.426969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.428391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:00.428962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.432646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.445064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1648324e8b134098a0365bcde058e289'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:00.465554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 913/1810] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:00.467813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.469944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1648324e8b134098a0365bcde058e289'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:00.470955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1648324e8b134098a0365bcde058e289'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:00.488692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d2b9-a779-4335-aa50-2fbeea7dbe09 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.497478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.497707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.498423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.498905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.500144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.502647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.516610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 914/1811] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:00.517061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.520273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:00.544135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e75642-bc4f-45b4-8416-0538b7ea3eec admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.553570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.555555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.556159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.556742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] PATH_INFO: `/v3/projects/060a66502d93405c8b2aa01d16ac7d8d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.559271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060a66502d93405c8b2aa01d16ac7d8d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.563192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:00.603844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1648324e8b134098a0365bcde058e289'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:00.618114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 898/1812] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:00 2022] DELETE /identity/v3/users/1648324e8b134098a0365bcde058e289 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:00.618919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:00.621114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:00.790621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9b2a-2215-4b70-93c8-d22cd641f370 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:00.790621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:00.790621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:00.790621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:00.790982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:00.794094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:00.798504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:01.313463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060a66502d93405c8b2aa01d16ac7d8d', 'request_id': 'req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:01.326819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060a66502d93405c8b2aa01d16ac7d8d', 'request_id': 'req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:01.337260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060a66502d93405c8b2aa01d16ac7d8d', 'request_id': 'req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:01.751557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 899/1813] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:00 2022] POST /identity/v3/projects => generated 363 bytes in 1133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:01.753623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:01.760089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:01.778104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] The token cache is being invalidate because project 060a66502d93405c8b2aa01d16ac7d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:01.778477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:01.779295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f80a1fa6-9706-4c34-a5e2-57b02663f4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:01.781916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 915/1814] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:00 2022] DELETE /identity/v3/projects/060a66502d93405c8b2aa01d16ac7d8d => generated 0 bytes in 1265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:01.784381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:01.785497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:01.785780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:01.831651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:01.833313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:01.880358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8723e9-8394-4aaa-85dc-81c9072c7c68 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:01.892570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:01.894570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:01.895336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:01.895906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] PATH_INFO: `/v3/projects/509825b6bcb24e5dbc39bcf09987302a/users/1f647a1e051c496ba15e64885f0558de/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:01.901370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509825b6bcb24e5dbc39bcf09987302a, user_id=1f647a1e051c496ba15e64885f0558de, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:01.910021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:01.930491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 900/1815] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:01 2022] PUT /identity/v3/projects/509825b6bcb24e5dbc39bcf09987302a/users/1f647a1e051c496ba15e64885f0558de/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:01.935755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:01.939897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:02.000817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 916/1816] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:02.002638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:02.005611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:02.042298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ac87c-4f1f-45d2-8ad6-f143f6c7bec0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.096241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85036fc8-5a01-41f0-88f7-921b010c0866 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.105199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.106336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.106816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.107949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.110544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.114364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.163125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.164071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.164362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.164629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] PATH_INFO: `/v3/users/1cfb96389c454624b6749856a5417afd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.166700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cfb96389c454624b6749856a5417afd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.168763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.246748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 901/1817] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:01 2022] POST /identity/v3/users => generated 282 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:02.248011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:02.251271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:02.371659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a746585-dcab-4d86-b0ca-f73d418be158 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.381619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.382846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.386361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.386361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.401836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.406488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.427550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 902/1818] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:02.429045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:02.448165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3aa6-2e0b-4e4b-a63e-9b9af680d80e service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.457866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.463988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.463988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.463988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.463988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.465799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.490634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 903/1819] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:02.499100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:02.502570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cfb96389c454624b6749856a5417afd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:02.506341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cfb96389c454624b6749856a5417afd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:02.520339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf93a0e-0e60-4385-8175-3360dbdd1002 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.544641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.544641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.544641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.544641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.545403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.545519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cfb96389c454624b6749856a5417afd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:02.545660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.552960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cfb96389c454624b6749856a5417afd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:02.582998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 904/1820] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:02.583188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5794ed78-5e69-42be-87d8-b1f6ba27b1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cfb96389c454624b6749856a5417afd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:02.583346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:02.587905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:02.617680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4e213-66b3-4abf-bf8b-d74c88901e0f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:02.630061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:02.632502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.633080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.633690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] PATH_INFO: `/v3/projects/f37a7d870c69485db74890cc7db28d24/users/605ac9756c894c3d8eb63debcf7f1619/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.670675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37a7d870c69485db74890cc7db28d24, user_id=605ac9756c894c3d8eb63debcf7f1619, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:02.676531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:02.707377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 917/1821] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:02 2022] DELETE /identity/v3/users/1cfb96389c454624b6749856a5417afd => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:02.846949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 905/1822] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:02 2022] PUT /identity/v3/projects/f37a7d870c69485db74890cc7db28d24/users/605ac9756c894c3d8eb63debcf7f1619/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:02.854802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:02.856010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:02.856010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:02.934411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] MFA Rules not processed for user `605ac9756c894c3d8eb63debcf7f1619`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:02.936084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:03.027005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 918/1823] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:03.034009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:03.197969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edc589-d262-40f1-8885-e1bf056ff7b6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:03.208551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:03.210607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:03.213268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:03.213975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:03.217736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:03.220883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:03.346001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 906/1824] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:03 2022] POST /identity/v3/projects => generated 365 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:03.352658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:03.375799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006a359-f939-41eb-83e1-a74ec32448af None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:03.383763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:03.388035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:03.573149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:03.574283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:03.575036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:03.575496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:03.576782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:03.581600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:03.582293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909424cd-b6a9-48af-b963-adeca9511a2b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:03.589357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:03.591442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:03.592738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:03.592738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] PATH_INFO: `/v3/projects/aaf2855f72284b18bc517517722493ca` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:03.595043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf2855f72284b18bc517517722493ca)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:03.599172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:03.707329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 919/1825] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:03 2022] POST /identity/v3/users => generated 289 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:03.707329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:03.721822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:03.815338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf2855f72284b18bc517517722493ca', 'request_id': 'req-3666884b-26ad-4c75-96c5-5c808a9798b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:03.817654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4da9-7d81-446c-9ef3-ace13efea4d7 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:03.828226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf2855f72284b18bc517517722493ca', 'request_id': 'req-3666884b-26ad-4c75-96c5-5c808a9798b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:03.840911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf2855f72284b18bc517517722493ca', 'request_id': 'req-3666884b-26ad-4c75-96c5-5c808a9798b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:03.929159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:03.930241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:03.930769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:03.931267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] PATH_INFO: `/v3/projects/9f23c513faf24335bf5095c78dfb9902` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:03.933725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f23c513faf24335bf5095c78dfb9902)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:03.938501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:03.998895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:04.020690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] The token cache is being invalidate because project aaf2855f72284b18bc517517722493ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:04.022951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.023765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.031977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 907/1826] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:03 2022] DELETE /identity/v3/projects/aaf2855f72284b18bc517517722493ca => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:04.036440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:04.042095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:04.077727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666884b-26ad-4c75-96c5-5c808a9798b4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:04.088824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:04.089996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.090619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.091137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.094792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:04.098291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:04.121538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 908/1827] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:04.127139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:04.136611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:04.200795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede87d-1b01-4620-a183-626ceaf92fab admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:04.204886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f23c513faf24335bf5095c78dfb9902', 'request_id': 'req-d22a142f-5fcb-4341-8e4c-a6406b056ff4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.225636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f23c513faf24335bf5095c78dfb9902', 'request_id': 'req-d22a142f-5fcb-4341-8e4c-a6406b056ff4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.237918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f23c513faf24335bf5095c78dfb9902', 'request_id': 'req-d22a142f-5fcb-4341-8e4c-a6406b056ff4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.350703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:04.351696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.352121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.352508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] PATH_INFO: `/v3/users/dfcd2a135d9640e29535bdfbdf4a1cb9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.354448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfcd2a135d9640e29535bdfbdf4a1cb9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:04.356893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:04.372374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:04.384234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] The token cache is being invalidate because project 9f23c513faf24335bf5095c78dfb9902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:04.384739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.386138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.388999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 920/1828] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:03 2022] DELETE /identity/v3/projects/9f23c513faf24335bf5095c78dfb9902 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:04.390276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:04.392395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:04.412122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a142f-5fcb-4341-8e4c-a6406b056ff4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:04.420034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:04.421154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.422378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.422845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] PATH_INFO: `/v3/projects/d8361204e269445c95c708c2092472d3/users/35a246802eec468bb55167b77816d32d/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.452315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8361204e269445c95c708c2092472d3, user_id=35a246802eec468bb55167b77816d32d, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:04.459541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34747f-d047-4643-9705-71735a950936 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:04.594919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 921/1829] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:04 2022] PUT /identity/v3/projects/d8361204e269445c95c708c2092472d3/users/35a246802eec468bb55167b77816d32d/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:04.600860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd09-849a-44d7-88b2-fcad1791e024 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.601351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd09-849a-44d7-88b2-fcad1791e024 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.602212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd09-849a-44d7-88b2-fcad1791e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.662225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2d60cd09-849a-44d7-88b2-fcad1791e024 None None] MFA Rules not processed for user `1f647a1e051c496ba15e64885f0558de`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:04.665797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2d60cd09-849a-44d7-88b2-fcad1791e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:04.673666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfcd2a135d9640e29535bdfbdf4a1cb9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.685599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfcd2a135d9640e29535bdfbdf4a1cb9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.717694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfcd2a135d9640e29535bdfbdf4a1cb9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.719756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfcd2a135d9640e29535bdfbdf4a1cb9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.792023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 922/1830] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:04.793580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bd58-d970-43e2-9ad3-3c92ae40c41b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.793873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bd58-d970-43e2-9ad3-3c92ae40c41b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.794151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826bd58-d970-43e2-9ad3-3c92ae40c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.813786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-05bda053-b5b0-4e85-83ae-d23e64c7ea6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfcd2a135d9640e29535bdfbdf4a1cb9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:04.831128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 909/1831] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:04 2022] DELETE /identity/v3/users/dfcd2a135d9640e29535bdfbdf4a1cb9 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:04.834941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:04.835372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:04.835710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0826bd58-d970-43e2-9ad3-3c92ae40c41b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:04.835826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:04.838594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0826bd58-d970-43e2-9ad3-3c92ae40c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:04.845141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 910/1832] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:37:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:37:04.849999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:04.849999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.029376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62344fe-d98d-4171-9a61-abf2a0e61217 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:05.034091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 923/1833] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:05.037279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.037880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.038569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.067101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:05.068592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.069351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.070060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.071779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:05.075404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b8528-6781-47e6-92b1-8463017febb6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:05.079682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:05.082067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.103345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 911/1834] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:05.105525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.105810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.106426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.131890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:05.133164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.155614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 912/1835] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:05.156427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:05.175449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7db9f-2480-4f37-a60d-aec68b6d5971 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:05.184929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:05.186521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.187564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.203914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] PATH_INFO: `/v3/projects/d8361204e269445c95c708c2092472d3/users/35a246802eec468bb55167b77816d32d/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.210911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8361204e269445c95c708c2092472d3, user_id=35a246802eec468bb55167b77816d32d, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:05.216330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba568a31-49c9-4826-b88d-33fe3b985f35 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:05.235660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 924/1836] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:05.237291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:05.256157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 913/1837] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:05 2022] PUT /identity/v3/projects/d8361204e269445c95c708c2092472d3/users/35a246802eec468bb55167b77816d32d/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:05.260108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d44e3b-0bb0-4ecc-9bb6-93d20047a4e9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:05.284220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.284220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.284220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.346871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] MFA Rules not processed for user `35a246802eec468bb55167b77816d32d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:05.356865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.420494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:05.423110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.423695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.424373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.427325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:05.427866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:05.432559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:05.468173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 925/1838] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:05.476116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:05.528024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe763-9d79-45dd-a287-e9050dac40ac admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:05.581995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:05.588206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:05.588489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:05.589036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:05.591663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:05.596011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:05.626588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 914/1839] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:05.638444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:05.641125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.798116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 926/1840] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:05 2022] POST /identity/v3/projects => generated 399 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:05.799788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:05.809502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:05.878541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4461b8-ab74-4277-837b-737a04e8e663 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:05.895604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:06.107755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89f55-18ae-4e37-8ae2-109c49c8adda admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:06.122679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:06.123982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.124470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.124947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.136415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:06.141788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.142295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.142772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.180277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:06.183451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:06.228746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 915/1841] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:06.230122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:06.326819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:06.331329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:06.396345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 927/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:06.399422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:06.408629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:06.424131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426fa2-41e2-4999-8d20-6776002a89f7 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:06.437922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:06.441658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.442734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.442734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.444426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:06.451900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07174a5-d7e7-4416-8903-b00eb8139d58 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:06.489108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e88c5-d239-44e4-98ea-1c60a55414f0 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:06.512602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:06.516519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.518641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.519615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] PATH_INFO: `/v3/projects/6eb199bc06c84188b9fd09140b97a33f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.521810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb199bc06c84188b9fd09140b97a33f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:06.524312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:06.629600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 916/1843] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:06 2022] POST /identity/v3/users => generated 300 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:06.633627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8279-a10e-4e64-af38-fb4644469ecf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.633627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8279-a10e-4e64-af38-fb4644469ecf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.633627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8279-a10e-4e64-af38-fb4644469ecf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.643899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 917/1844] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:37:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:37:06.644862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8279-a10e-4e64-af38-fb4644469ecf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:06.691687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8279-a10e-4e64-af38-fb4644469ecf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:06.697783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:06.698986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.699505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.701724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.703395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:06.705474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6cb60-0d38-4ff4-b2c8-f80228a30cbe admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:06.745535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 918/1845] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:06.745535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:06.745535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:06.748114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:06.782461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb199bc06c84188b9fd09140b97a33f', 'request_id': 'req-3ac712bc-c208-49c8-a5a6-922a02b6d957'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:06.824500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb199bc06c84188b9fd09140b97a33f', 'request_id': 'req-3ac712bc-c208-49c8-a5a6-922a02b6d957'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:06.846292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:06.849971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:06.851051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb199bc06c84188b9fd09140b97a33f', 'request_id': 'req-3ac712bc-c208-49c8-a5a6-922a02b6d957'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:06.896424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:06.954788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 919/1846] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:06.956352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:06.985507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] The token cache is being invalidate because project 6eb199bc06c84188b9fd09140b97a33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:06.986161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:06.988244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:06.996800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 928/1847] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:06 2022] DELETE /identity/v3/projects/6eb199bc06c84188b9fd09140b97a33f => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:07.005643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.008640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.044726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246448b-cd6b-426b-9d41-d327a917ea35 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.051227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.052045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.052405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.052750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.054314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.056674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.071170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 920/1848] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:07.072333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.078216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.098907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686910c-47e9-4214-b03d-f6164ed39a68 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.106538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.107327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.107890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.108243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.111497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.128465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.130322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4928-74db-46a6-9283-3e0a223e0280 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.141913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 921/1849] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:07.144710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.145096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.145337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.159337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac712bc-c208-49c8-a5a6-922a02b6d957 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.165569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.166755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.167222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.167855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.172265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.183102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:07.185230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.201144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 922/1850] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:07.203391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.206532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.226130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fce4c8-4cea-4cec-9ed5-4494b688c185 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.231800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.232926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.233209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.233482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] PATH_INFO: `/v3/projects/e95d419aa3524163a9efedeae7b6437b/users/d3493362cec04173ade3a97ae500bd51/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.253457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95d419aa3524163a9efedeae7b6437b, user_id=d3493362cec04173ade3a97ae500bd51, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.258265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c8879-fa34-4807-8895-417e18408aa0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.292133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.293629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73b0ba-768d-4bb7-89bc-8cf1ab0f5cf0 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.305226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 929/1851] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:07.306530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.306814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.307112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.325991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:07.327270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.335900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 930/1852] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:07.340818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.354772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348779e7-58c0-40fb-b50e-8e677373112d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.363355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.364040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.364923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.365487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.367729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:07.368778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.375420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:07.382273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 923/1853] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:07 2022] PUT /identity/v3/projects/e95d419aa3524163a9efedeae7b6437b/users/d3493362cec04173ade3a97ae500bd51/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:07.389204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.394920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.395183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.395470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.426262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 931/1854] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:37:07.429784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.439892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] MFA Rules not processed for user `d3493362cec04173ade3a97ae500bd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:07.441862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:07.449354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bc5f2-d2b9-41a8-8d8b-adcdefb2677f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.574784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.575727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.576205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.576660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.578724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.580768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.590274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 924/1855] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:07.598630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.620242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bfa81-1609-4774-a0f8-e511c027660b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.632136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.633298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.633820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.634351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.635648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.639351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:07.726299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 932/1856] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:07.732373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.770563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 925/1857] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/projects => generated 401 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:07.776593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:07.885373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c0bc-082b-4124-afab-70ef56bf098e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.920341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075db70-8e3d-4594-90b6-e8695ab19b2f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:07.931145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:07.932480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:07.933188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:07.934233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:07.935022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:07.939735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.081506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 926/1858] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/users => generated 307 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:08.086442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.094242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:08.094505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.095613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.096245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.096764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.098337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.107806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.134618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ff380-9218-4d4b-a8c9-8cf3d2e03d75 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.140226 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.141036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.141323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.141617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] PATH_INFO: `/v3/users/d241bafb09094103885a4389a4b70050` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.143298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d241bafb09094103885a4389a4b70050)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.145673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.300193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 933/1859] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:07 2022] POST /identity/v3/users => generated 294 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:08.301634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.319909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7a0e8-b43f-46e1-a119-89e4d03d62b8 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.326613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.327294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.328329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.328791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.329552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.331263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.344837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 934/1860] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:08.346483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.366465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c6433-c8a3-4a74-b954-b47601cbacda admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.373850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.375102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.375591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.376307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.377751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.381123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.398711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 935/1861] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:08.399908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.414013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d9b58-35c1-4cd0-a06e-f9481495d0f1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.419665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.420722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.421060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.421490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] PATH_INFO: `/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae/users/9553e369b68044f1b3cc39080f4d9932/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.450843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f41df9deab349fda6b0eb33cff0a7ae, user_id=9553e369b68044f1b3cc39080f4d9932, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.454342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd241bafb09094103885a4389a4b70050'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:08.456623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.466836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd241bafb09094103885a4389a4b70050'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:08.487202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd241bafb09094103885a4389a4b70050'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:08.488465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd241bafb09094103885a4389a4b70050'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:08.585715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 936/1862] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:08 2022] PUT /identity/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae/users/9553e369b68044f1b3cc39080f4d9932/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:08.597496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.629617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd241bafb09094103885a4389a4b70050'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:08.639311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 927/1863] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:08 2022] DELETE /identity/v3/users/d241bafb09094103885a4389a4b70050 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:08.641571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.645353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:08.695229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820a41-97aa-4023-aab0-16b09c0d39ef admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.767377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e03e-67fb-4758-bdb6-245dc2b5ec72 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.776784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.777760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.778356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.778774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.779737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.782009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.801760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 928/1864] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:08.815419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.826752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.828540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.829029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.829740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.832014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.837038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.850886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e46b4e-2f2e-4115-8de1-089e68547ca4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.857746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:08.865966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 937/1865] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:08.867178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:08.868914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.869500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.870068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] PATH_INFO: `/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae/users/9553e369b68044f1b3cc39080f4d9932/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:08.872916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:08.877391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f41df9deab349fda6b0eb33cff0a7ae, user_id=9553e369b68044f1b3cc39080f4d9932, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:08.877643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:08.886881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3df5bd-1058-45db-a01b-7b5df508916b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:08.906760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb881-9e9a-4e3a-b784-a30af0ba4a57 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:08.918770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:08.928662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 929/1866] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:08 2022] PUT /identity/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae/users/9553e369b68044f1b3cc39080f4d9932/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:08.953379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:08.953379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:08.953379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.002389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] MFA Rules not processed for user `9553e369b68044f1b3cc39080f4d9932`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:09.008088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.024728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:09.026425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.026937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.027440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] PATH_INFO: `/v3/projects/3625f08cd9e2427f919356ad88e6a067/users/2adfd5721c7f4e8ba8727807c2df77af/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.053116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3625f08cd9e2427f919356ad88e6a067, user_id=2adfd5721c7f4e8ba8727807c2df77af, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:09.056985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:09.129710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 930/1867] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:09.129710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:09.129710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.144620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582afa81-57f2-4e3f-a5bc-36ffbc477fe6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:09.147136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 938/1868] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:08 2022] PUT /identity/v3/projects/3625f08cd9e2427f919356ad88e6a067/users/2adfd5721c7f4e8ba8727807c2df77af/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:09.156166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:09.159837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.256887 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:09.258637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.258923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.259264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] PATH_INFO: `/v3/projects/f72038fab1624d6285967d66f2e8098c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.260896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f72038fab1624d6285967d66f2e8098c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:09.263193 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:09.313308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695505c-b789-4acf-b544-544bd1014b94 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:09.320834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:09.322150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.322750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.323299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.409741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f72038fab1624d6285967d66f2e8098c', 'request_id': 'req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:09.420621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f72038fab1624d6285967d66f2e8098c', 'request_id': 'req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:09.429519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f72038fab1624d6285967d66f2e8098c', 'request_id': 'req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:09.475716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:09.478638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d11366-7145-498d-a888-fda7955965c2 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:09.498262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 939/1869] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:09.501441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.502518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.502782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.518278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] The token cache is being invalidate because project f72038fab1624d6285967d66f2e8098c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:09.518571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:09.522248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:09.530504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 931/1870] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:09 2022] DELETE /identity/v3/projects/f72038fab1624d6285967d66f2e8098c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:09.532312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:09.534257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.566648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] MFA Rules not processed for user `2adfd5721c7f4e8ba8727807c2df77af`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:09.570071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.673264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 940/1871] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:09.682547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:09.691068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:09.736685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83ef7-bea6-4672-ae86-8ae5ee3b6fa2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:09.759801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:09.762050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.762638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.763272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] PATH_INFO: `/v3/users/1f647a1e051c496ba15e64885f0558de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.766354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f647a1e051c496ba15e64885f0558de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:09.770234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:09.836655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f078e29-f795-4afa-8371-19855ab86404 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:09.848176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:09.849704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:09.850520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:09.851254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:09.852740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:09.856882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:09.973879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 941/1872] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:09 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:09.973879 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:09.975183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:10.020626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f647a1e051c496ba15e64885f0558de'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.034676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f647a1e051c496ba15e64885f0558de'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.048998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710703b-bcef-409a-9249-97974938d0e6 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.054622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.055347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.055725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.056051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.058836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f647a1e051c496ba15e64885f0558de'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.059025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:10.060557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f647a1e051c496ba15e64885f0558de'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.149825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f647a1e051c496ba15e64885f0558de'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.160898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 932/1873] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:09 2022] DELETE /identity/v3/users/1f647a1e051c496ba15e64885f0558de => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:10.160898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:10.163420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:10.168651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.171342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.198709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 942/1874] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:10.319934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc75af-6dd3-4f0e-96ff-a04954f4fb7f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.328416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.329387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.329731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.330010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.332000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.335383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.450984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 933/1875] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:10 2022] POST /identity/v3/users => generated 300 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:10.456210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:10.534689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e724b1-089f-4f10-9545-bebf2496dbe1 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.547288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.548640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.549012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.549364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.550420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.552313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.560854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:10.575594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:10.577622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 943/1876] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:10.579253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:10.580286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:10.601414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b376-4999-41be-9139-27003a84d69d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.609893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e3873-f73a-4a36-910f-35891e6753bd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.612734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.614336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.614936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.615447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] PATH_INFO: `/v3/projects/0ad65934f61e4335b59f7189d24921e0/users/dd5a7f439e5a42a3a7d09a9e01be731b/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.634662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.634662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.634662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.634662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] PATH_INFO: `/v3/projects/509825b6bcb24e5dbc39bcf09987302a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.635204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509825b6bcb24e5dbc39bcf09987302a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.635664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.658973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad65934f61e4335b59f7189d24921e0, user_id=dd5a7f439e5a42a3a7d09a9e01be731b, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.665392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.666601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:10.751012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 944/1877] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:10 2022] PUT /identity/v3/projects/0ad65934f61e4335b59f7189d24921e0/users/dd5a7f439e5a42a3a7d09a9e01be731b/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:10.757320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:10.831805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509825b6bcb24e5dbc39bcf09987302a', 'request_id': 'req-4ae9732e-6d20-4460-abcd-7c2901529a44'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.850908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509825b6bcb24e5dbc39bcf09987302a', 'request_id': 'req-4ae9732e-6d20-4460-abcd-7c2901529a44'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.871092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509825b6bcb24e5dbc39bcf09987302a', 'request_id': 'req-4ae9732e-6d20-4460-abcd-7c2901529a44'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.892160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e6b0d-8223-4aa1-b049-4abed1529e25 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:10.988970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:10.990189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:10.991211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:10.996387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:10.996387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:10.996610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] The token cache is being invalidate because project 509825b6bcb24e5dbc39bcf09987302a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:10.996610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:10.996954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:10.999422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4ae9732e-6d20-4460-abcd-7c2901529a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:11.004188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 934/1878] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:10 2022] DELETE /identity/v3/projects/509825b6bcb24e5dbc39bcf09987302a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:11.016745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67178f4-b8b1-4c76-af23-4d1506b25210 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.017157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67178f4-b8b1-4c76-af23-4d1506b25210 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.017441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67178f4-b8b1-4c76-af23-4d1506b25210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.033504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 945/1879] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:11.040813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.046964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:11.048676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f67178f4-b8b1-4c76-af23-4d1506b25210 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:11.049872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f67178f4-b8b1-4c76-af23-4d1506b25210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:11.061404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 935/1880] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:11.068920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.069486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.069909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.087599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba296d-1e67-4711-864f-6a8c9f01f571 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.093884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:11.095854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:11.102930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.104471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.106622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.106934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] PATH_INFO: `/v3/projects/0ad65934f61e4335b59f7189d24921e0/users/dd5a7f439e5a42a3a7d09a9e01be731b/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.112858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad65934f61e4335b59f7189d24921e0, user_id=dd5a7f439e5a42a3a7d09a9e01be731b, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.113171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 936/1881] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:11.114824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.117944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f015-ae2a-4b33-aeaa-c13de0d12add admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.130154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04ad3b-14e5-468d-845f-2816c1848418 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.153878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 946/1882] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:11 2022] PUT /identity/v3/projects/0ad65934f61e4335b59f7189d24921e0/users/dd5a7f439e5a42a3a7d09a9e01be731b/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:11.170278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.170675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.171665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.248248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.249113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.249509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.250708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.251789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:11.252260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.254489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.264666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] MFA Rules not processed for user `dd5a7f439e5a42a3a7d09a9e01be731b`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:11.268393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:11.275162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 937/1883] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:11.282690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.302505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda878a-9610-45c5-b54d-bfd1c9e674c2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.310960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.311406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.311940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.312434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.313984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.317973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.398049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 947/1884] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:11.416252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.418768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:11.488160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 938/1885] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:11 2022] POST /identity/v3/projects => generated 357 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:11.495080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.543435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb8c42-a71d-435c-9d5f-228161f54e2a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.642575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e452ff-50c2-45d4-afb7-50ead5d0cd83 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.650302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.651709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.652033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.652355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.652957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.654106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.654789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.655657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.657574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.663611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.766915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.771020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.779802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 939/1886] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:11 2022] POST /identity/v3/users => generated 279 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:11.786269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.799433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 948/1887] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:11.847050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d1e56-e6f6-42ac-8eae-890c2da8bd24 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.868419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.869078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.869445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.869780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.871021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.872871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.890443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 940/1888] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:11.902009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:11.915999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed372d1-8057-46c8-83d8-deee2334c22a service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:11.923214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:11.924657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:11.926107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:11.927331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:11.928743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:11.931614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7d954-8955-448e-8066-2a26aa3ba857 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:11.952924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 949/1889] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:11.958740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:12.001205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e8ed2-0b42-4c1f-85b0-64901420d330 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:12.012499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:12.015922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:12.016814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:12.017263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] PATH_INFO: `/v3/projects/a9d76992788b4e108127bb3a5177fa30/users/91bbd7e313394e04b2aaf81dd91612c9/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:12.072470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d76992788b4e108127bb3a5177fa30, user_id=91bbd7e313394e04b2aaf81dd91612c9, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:12.077723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:12.168110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 941/1890] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:11 2022] PUT /identity/v3/projects/a9d76992788b4e108127bb3a5177fa30/users/91bbd7e313394e04b2aaf81dd91612c9/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:12.183404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:12.185516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:12.186151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:12.226551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] MFA Rules not processed for user `91bbd7e313394e04b2aaf81dd91612c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:12.227673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:12.413677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 950/1891] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:12.420448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:12.526433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:12.528417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:12.569508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76218474-66c0-497f-b500-fd3fb529a33a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:12.579315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:12.580508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:12.580962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:12.581415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:12.582696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:12.586409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:12.692165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989bd9f-7291-4b0f-a0c6-2f61f77eac1d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:12.698657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:12.699824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:12.700220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:12.700538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] PATH_INFO: `/v3/users/605ac9756c894c3d8eb63debcf7f1619` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:12.702896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] RBAC: Authorizing `identity:delete_user(user_id=605ac9756c894c3d8eb63debcf7f1619)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:12.707299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:12.751489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 942/1892] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:12 2022] POST /identity/v3/projects => generated 353 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:12.756344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:12.805919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:12.871890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:12.882909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42ece-dd24-432c-8e90-7638721aaecc admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:12.896395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:12.898284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:12.899038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:12.900409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:12.902179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:12.907066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:12.954313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605ac9756c894c3d8eb63debcf7f1619'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:12.955523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605ac9756c894c3d8eb63debcf7f1619'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:12.979445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605ac9756c894c3d8eb63debcf7f1619'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:12.980711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605ac9756c894c3d8eb63debcf7f1619'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:12.997113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605ac9756c894c3d8eb63debcf7f1619'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.033258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 943/1893] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:12 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:13.034989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.038527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.061993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4db707-b35c-4a32-9df6-6c6edb26fa56 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.071148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.072622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.073124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.073647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] PATH_INFO: `/v3/users/2adfd5721c7f4e8ba8727807c2df77af` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.076121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2adfd5721c7f4e8ba8727807c2df77af)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.079954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:13.115114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 951/1894] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:12 2022] DELETE /identity/v3/users/605ac9756c894c3d8eb63debcf7f1619 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:13.117276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.121210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.251749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99abd50-1b3c-4000-b44c-3a081a0a489d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.259429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.260486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.260932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.261381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.262671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.265596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae688-cde0-4404-a861-c38865f0c532 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:13.288699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 952/1895] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:13.288699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.288699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.288699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.319356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:13.320671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.331731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 953/1896] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:13.333023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.352806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98b05-3fbf-4d4b-90f0-3372c13ee272 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.368420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.370051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.370599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.371130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] PATH_INFO: `/v3/projects/09ca605da9fa46bda3362c53f34f1c76/users/0d514aeb7ce148f189d167b1dced14dc/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.401140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2adfd5721c7f4e8ba8727807c2df77af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.413905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ca605da9fa46bda3362c53f34f1c76, user_id=0d514aeb7ce148f189d167b1dced14dc, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.417106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:13.428240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2adfd5721c7f4e8ba8727807c2df77af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.459988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2adfd5721c7f4e8ba8727807c2df77af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.461383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2adfd5721c7f4e8ba8727807c2df77af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.574145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2adfd5721c7f4e8ba8727807c2df77af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:13.589892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 954/1897] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:13 2022] PUT /identity/v3/projects/09ca605da9fa46bda3362c53f34f1c76/users/0d514aeb7ce148f189d167b1dced14dc/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:13.597390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.600558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.610959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 944/1898] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:13 2022] DELETE /identity/v3/users/2adfd5721c7f4e8ba8727807c2df77af => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:13.615164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.623495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.769979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e17c3e-6bad-4595-89bd-7c7ae849e428 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.847106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b689f4-c15b-4b85-82ea-f7685d7c573f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.862786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.863497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.863863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.864206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.865241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.867323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac52952-cbd8-48f5-8d2a-c31db91133c0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:13.890457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 945/1899] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:13.890762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.890929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.891061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.891192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.891192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.891475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.891475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.893357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.924474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:13.927092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:13.933297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.937184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:13.944715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 946/1900] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:13.946346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.968023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 955/1901] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:13.969613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410fdd2-41ce-4840-a8c5-59fd555dadd5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:13.977302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:13.978739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:13.979319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:13.979630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] PATH_INFO: `/v3/projects/09ca605da9fa46bda3362c53f34f1c76/users/0d514aeb7ce148f189d167b1dced14dc/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:13.983211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ca605da9fa46bda3362c53f34f1c76, user_id=0d514aeb7ce148f189d167b1dced14dc, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:13.990167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:13.991491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fd821-2faa-488c-b0b1-42e6026946c2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:14.027477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 947/1902] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:13 2022] PUT /identity/v3/projects/09ca605da9fa46bda3362c53f34f1c76/users/0d514aeb7ce148f189d167b1dced14dc/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:14.030860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:14.031386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:14.032099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:14.081608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edfe5b-1ea3-4a40-9411-b8d7efeb8f88 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:14.116014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] MFA Rules not processed for user `0d514aeb7ce148f189d167b1dced14dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:14.118661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:14.168467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:14.261199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:14.262029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:14.262166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:14.262533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:14.309739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 948/1903] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:14.362974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:14.365323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:14.379632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:14.383968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:14.399272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 956/1904] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:14.458214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71636221-9b05-4cb1-8b35-dd999bdfe91b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:14.462684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:14.463658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:14.463938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:14.464357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:14.475043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:14.476511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:14.488865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 949/1905] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:14 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:14.508770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:14.943205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:14.945166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:14.964327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424bf33-9678-4951-a019-156b256e63fc service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:14.971892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:14.974084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:14.975168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:14.984211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] PATH_INFO: `/v3/projects/f37a7d870c69485db74890cc7db28d24` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:14.984211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37a7d870c69485db74890cc7db28d24)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:14.984211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:15.222356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:15.224666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:15.247783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37a7d870c69485db74890cc7db28d24', 'request_id': 'req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.263906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b010c-02ec-40a2-82b6-c5b985e51c96 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:15.265835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37a7d870c69485db74890cc7db28d24', 'request_id': 'req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.279932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37a7d870c69485db74890cc7db28d24', 'request_id': 'req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.399214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:15.399871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:15.400353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:15.402448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:15.406466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:15.537721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] The token cache is being invalidate because project f37a7d870c69485db74890cc7db28d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:15.538227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.539292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.542506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 957/1906] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:14 2022] DELETE /identity/v3/projects/f37a7d870c69485db74890cc7db28d24 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:15.550378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:15.552689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:15.553954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:15.555653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:15.603587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 950/1907] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:15.687849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6235c1-a0d1-4156-b8a1-6a8e9c6f6a6b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:15.695323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:15.696435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:15.696834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:15.697194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] PATH_INFO: `/v3/users/35a246802eec468bb55167b77816d32d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:15.699139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a246802eec468bb55167b77816d32d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:15.701384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:15.962442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:15.984104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a246802eec468bb55167b77816d32d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:15.985765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a246802eec468bb55167b77816d32d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.005883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:16.012268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:16.019792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a246802eec468bb55167b77816d32d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.021267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a246802eec468bb55167b77816d32d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.037810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a246802eec468bb55167b77816d32d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.065054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:16.085822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8e80-8aee-4696-abb9-381fc9aba1c1 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:16.097234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:16.097404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:16.098224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:16.098974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] PATH_INFO: `/v3/projects/3625f08cd9e2427f919356ad88e6a067` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:16.101400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3625f08cd9e2427f919356ad88e6a067)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:16.105016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:16.161504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 958/1908] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:15 2022] DELETE /identity/v3/users/35a246802eec468bb55167b77816d32d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:16.309996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3625f08cd9e2427f919356ad88e6a067', 'request_id': 'req-58c1fed3-6e9e-4085-9067-482ff8de628d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.323274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3625f08cd9e2427f919356ad88e6a067', 'request_id': 'req-58c1fed3-6e9e-4085-9067-482ff8de628d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.338736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3625f08cd9e2427f919356ad88e6a067', 'request_id': 'req-58c1fed3-6e9e-4085-9067-482ff8de628d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.555397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] The token cache is being invalidate because project 3625f08cd9e2427f919356ad88e6a067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:16.556603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.557113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:16.558981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 951/1909] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:16 2022] DELETE /identity/v3/projects/3625f08cd9e2427f919356ad88e6a067 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:16.564215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:16.566662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:16.688957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67075852-4fe4-4393-b2c4-8ce28eea5620 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:16.799384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:16.800929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:16.801456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:16.801958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] PATH_INFO: `/v3/users/dd5a7f439e5a42a3a7d09a9e01be731b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:16.804412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5a7f439e5a42a3a7d09a9e01be731b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:16.809233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:16.859142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:16.862143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:16.884079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fed3-6e9e-4085-9067-482ff8de628d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:16.893003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:16.897321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:16.897321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:16.897321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] PATH_INFO: `/v3/projects/d8361204e269445c95c708c2092472d3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:16.907751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8361204e269445c95c708c2092472d3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:16.910072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:17.089042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8361204e269445c95c708c2092472d3', 'request_id': 'req-ec165408-944d-424b-9c76-541ceb85d2b0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.093597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5a7f439e5a42a3a7d09a9e01be731b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.095204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5a7f439e5a42a3a7d09a9e01be731b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.098076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8361204e269445c95c708c2092472d3', 'request_id': 'req-ec165408-944d-424b-9c76-541ceb85d2b0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.117187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8361204e269445c95c708c2092472d3', 'request_id': 'req-ec165408-944d-424b-9c76-541ceb85d2b0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.121919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5a7f439e5a42a3a7d09a9e01be731b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.124705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5a7f439e5a42a3a7d09a9e01be731b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.143565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f7029339-aedc-4322-b0b8-360ebcf526bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5a7f439e5a42a3a7d09a9e01be731b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.231669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] The token cache is being invalidate because project d8361204e269445c95c708c2092472d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:17.232150 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.233205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ec165408-944d-424b-9c76-541ceb85d2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:17.236635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 952/1910] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:16 2022] DELETE /identity/v3/projects/d8361204e269445c95c708c2092472d3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:17.246136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:17.246620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:17.247126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:17.252589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:17.292576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:17.295446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 959/1911] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:16 2022] DELETE /identity/v3/users/dd5a7f439e5a42a3a7d09a9e01be731b => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:17.297484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:17.471263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 953/1912] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:17.485826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:17.486943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:17.487811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:17.519448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:17.521887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:17.539839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 960/1913] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:17.554024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:17.592142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a2da6-ac61-4027-a947-e49985de3a2a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:17.604945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:17.606645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:17.608517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:17.609915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:17.611884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:17.613285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:17.617494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:17.655777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 954/1914] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:17.657901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:17.680555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da696e-f506-4eef-afab-3a34c0839de7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:17.690083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:17.691621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:17.692171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:17.692906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:17.694334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:17.700172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:17.852066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:17.889709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 961/1915] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:17 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:17.890335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:17.918016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54094b-64f1-4837-8728-851f2afc43a8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.126707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.128317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.130778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.131536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.133819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.147997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.283875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 955/1916] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:17 2022] POST /identity/v3/users => generated 294 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:18.289840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:18.307203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877aaa66-c48c-4eb5-ba47-6880a9809db7 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.316464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.317660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.318210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.318737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.320239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.323787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.349974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 962/1917] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:18.352097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:18.366895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98863bfe-2e6c-4751-98da-e62220d8bbf6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.373445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.374469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.374907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.375279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.376378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.378474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.392934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 956/1918] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:18.398796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:18.412776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4158cb-a427-4f92-9557-ee6d367bf0a4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.418838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.419790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.420453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.421547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] PATH_INFO: `/v3/projects/2349a41e9b2544aea5d31365db5ad521/users/ed77f93c28db4b27b5d0e29ba4b208eb/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.441551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2349a41e9b2544aea5d31365db5ad521, user_id=ed77f93c28db4b27b5d0e29ba4b208eb, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.444900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.469657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:18.473176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:18.517230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779194c-92f9-4d4d-9428-d4934cbd8204 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.552266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.552266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.552266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.552266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] PATH_INFO: `/v3/projects/0ad65934f61e4335b59f7189d24921e0` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.552772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad65934f61e4335b59f7189d24921e0)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.555516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.610917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 963/1919] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:18 2022] PUT /identity/v3/projects/2349a41e9b2544aea5d31365db5ad521/users/ed77f93c28db4b27b5d0e29ba4b208eb/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:18.616116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:18.618697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:18.730546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:18.734307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fe4cd-0dd0-4e0a-b353-44b6efd1224b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:18.744933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:18.746893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.747890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.748525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] PATH_INFO: `/v3/users/d3493362cec04173ade3a97ae500bd51` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:18.751068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3493362cec04173ade3a97ae500bd51)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:18.755680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:18.769687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad65934f61e4335b59f7189d24921e0', 'request_id': 'req-a3d8d814-433c-4bd1-a91a-86f3160aeeec'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:18.783536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad65934f61e4335b59f7189d24921e0', 'request_id': 'req-a3d8d814-433c-4bd1-a91a-86f3160aeeec'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:18.795780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad65934f61e4335b59f7189d24921e0', 'request_id': 'req-a3d8d814-433c-4bd1-a91a-86f3160aeeec'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:18.939748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] The token cache is being invalidate because project 0ad65934f61e4335b59f7189d24921e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:18.940517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:18.942554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a3d8d814-433c-4bd1-a91a-86f3160aeeec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:18.947936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 957/1920] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:18 2022] DELETE /identity/v3/projects/0ad65934f61e4335b59f7189d24921e0 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:18.950858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482c641-46b5-4bac-8037-3b9d744657f0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:18.952831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482c641-46b5-4bac-8037-3b9d744657f0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:18.957425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482c641-46b5-4bac-8037-3b9d744657f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.017314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1482c641-46b5-4bac-8037-3b9d744657f0 None None] MFA Rules not processed for user `ed77f93c28db4b27b5d0e29ba4b208eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:19.026404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1482c641-46b5-4bac-8037-3b9d744657f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.066319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3493362cec04173ade3a97ae500bd51'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:19.067782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3493362cec04173ade3a97ae500bd51'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:19.098303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3493362cec04173ade3a97ae500bd51'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:19.099869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3493362cec04173ade3a97ae500bd51'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:19.117780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3493362cec04173ade3a97ae500bd51'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:19.193463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 958/1921] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:19.193463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85540d0-81ff-4a0d-96a6-933f2d136683 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.193463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85540d0-81ff-4a0d-96a6-933f2d136683 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.193463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85540d0-81ff-4a0d-96a6-933f2d136683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.226039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 964/1922] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:18 2022] DELETE /identity/v3/users/d3493362cec04173ade3a97ae500bd51 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:19.231772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:19.232466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e85540d0-81ff-4a0d-96a6-933f2d136683 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:19.235812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.237468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e85540d0-81ff-4a0d-96a6-933f2d136683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.400659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 959/1923] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:19.402629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf6f1-5712-47b9-9ed0-f893d9ee068c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.402910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf6f1-5712-47b9-9ed0-f893d9ee068c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.403320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf6f1-5712-47b9-9ed0-f893d9ee068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.411601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e366f-a880-49ab-9e66-b65a7d8a63f9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:19.420886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:19.421667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.422376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.424353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.424353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:19.428330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:19.437086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f61bf6f1-5712-47b9-9ed0-f893d9ee068c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:19.441187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f61bf6f1-5712-47b9-9ed0-f893d9ee068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.466121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 960/1924] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:19.470947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.470947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.470947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.512987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:19.515915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.541663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 961/1925] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:19.548743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:19.552050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:19.646663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 965/1926] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/projects => generated 387 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:19.654792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:19.760012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd990b-7e83-465d-ad50-9be13a1aa86c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:19.783343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:19.841569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff330cb-50b5-45fa-b037-a7c0993a643a admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:19.850088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:19.851196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.851196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.851607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.852293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:19.852599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:19.854990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:19.868833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:19.869299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.869857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.870439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.882551 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 966/1927] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:19.883673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:19.887723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:19.889679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:19.897646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56326a-c9a9-4715-8e9f-93a16c9fa779 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:19.902898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:19.903780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.904132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.904948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.905694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 962/1928] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:19.906723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:19.908539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:19.912137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:19.925009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45a033-13e8-4615-8028-366f4d9ba870 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:19.934167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:19.935496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:19.936074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:19.936695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:19.938828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:19.942071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.010980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 967/1929] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:20.015390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.037938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a41a8-4696-4698-b493-433a8598d1f2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.047325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.049320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.049320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.049727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.051472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.055766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.076261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 963/1930] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:19 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:20.080575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 968/1931] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:20.083175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.097589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.135498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddb3d-36a8-4f71-941c-dcdde7fc251a admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.259744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.260538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.261031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.262807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.268989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.281375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.285339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900946d-177c-410e-9489-13ba0377348a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.302929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 969/1932] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:20.305088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.306234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.306763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.307252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.308872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.309106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.313655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.339180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdfc3-32ad-4a3f-8c36-f6fec437c69d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.367096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:20.371198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.372946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.373426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.373941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] PATH_INFO: `/v3/projects/dae2e3c472204e979f233ae454dda436/users/b0617d4fbbfb4ae0ad3359f4872450c4/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.414815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae2e3c472204e979f233ae454dda436, user_id=b0617d4fbbfb4ae0ad3359f4872450c4, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.419958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dee53-bc35-4c22-9086-0f385a051e14 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.515778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 964/1933] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:20 2022] POST /identity/v3/users => generated 287 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:20.521590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.535078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 970/1934] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:20 2022] PUT /identity/v3/projects/dae2e3c472204e979f233ae454dda436/users/b0617d4fbbfb4ae0ad3359f4872450c4/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:20.555267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.555573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.556766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.558162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a494-77ec-41f7-b44f-4acb0de0e317 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.646465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] MFA Rules not processed for user `b0617d4fbbfb4ae0ad3359f4872450c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:20.648029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:20.871852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 971/1935] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:20.884032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.891129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.892666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.894066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.896007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.897222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.901115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.928170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f25cc-7023-48e1-9bb0-be0ca87bd323 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.930390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 965/1936] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:20.935848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:20.965596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0a678-b050-4377-b6e6-1d0fc40870db admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:20.966832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.968663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.971530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.973568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.976533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.978695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:20.980904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:20.981198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:20.982653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:20.984606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:20.985016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:20.991032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:21.019554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 966/1937] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:21.023692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:21.045037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f76f2-0980-4bea-a5b6-ed475e497a5b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:21.068124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:21.069166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:21.072434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:21.072860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] PATH_INFO: `/v3/projects/6d2107609c264b7db70ffc4965ff45db/users/4f53a28588c24f1a8940c99be7e28917/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:21.112423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2107609c264b7db70ffc4965ff45db, user_id=4f53a28588c24f1a8940c99be7e28917, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:21.118578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85852d-d096-4163-8f5b-96c2afc0d3dc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:21.239844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 972/1938] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:20 2022] POST /identity/v3/projects => generated 385 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:21.241507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:21.249555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 967/1939] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:21 2022] PUT /identity/v3/projects/6d2107609c264b7db70ffc4965ff45db/users/4f53a28588c24f1a8940c99be7e28917/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:21.260044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:21.260403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:21.260988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:21.307613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:21.310668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:21.423206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3aa15-34d0-493c-bf4e-bb544215627e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:21.431000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:21.445518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:21.446737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:21.447225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:21.447700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:21.449050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:21.452797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:21.526436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 968/1940] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:21.528514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:21.532290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:21.554132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08040060-b746-4b70-a5ba-b47ebf8d16d0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:21.568214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:21.568214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:21.568214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:21.568214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] PATH_INFO: `/v3/projects/e95d419aa3524163a9efedeae7b6437b` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:21.569678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95d419aa3524163a9efedeae7b6437b)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:21.573663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:21.610845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 973/1941] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:21 2022] POST /identity/v3/users => generated 299 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:21.612385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:21.617373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:21.753900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41586e03-e84d-4560-a8ae-f39cc73805de admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:21.790773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95d419aa3524163a9efedeae7b6437b', 'request_id': 'req-ed42caf1-d917-4673-8b9c-996bd55539bb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:21.814752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95d419aa3524163a9efedeae7b6437b', 'request_id': 'req-ed42caf1-d917-4673-8b9c-996bd55539bb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:21.831924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95d419aa3524163a9efedeae7b6437b', 'request_id': 'req-ed42caf1-d917-4673-8b9c-996bd55539bb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:21.872419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:21.873493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:21.874082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:21.875117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:21.990563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] The token cache is being invalidate because project e95d419aa3524163a9efedeae7b6437b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:21.991166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:21.995446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:21.998614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.001648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.008215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 969/1942] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:21 2022] DELETE /identity/v3/projects/e95d419aa3524163a9efedeae7b6437b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:22.010571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:22.017517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:22.025484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 974/1943] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:22.028395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:22.054987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede23ff-a1fa-4918-9bc1-563a1a17c768 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:22.071067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42caf1-d917-4673-8b9c-996bd55539bb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:22.075081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:22.078729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.079670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.080344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] PATH_INFO: `/v3/users/9553e369b68044f1b3cc39080f4d9932` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.081676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:22.083488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9553e369b68044f1b3cc39080f4d9932)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.084239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.084871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.085533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.089232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.090056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.092950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.147513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 970/1944] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:22.149255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:22.172197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064dd3-a6b1-495f-8543-b654b35770e0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:22.184583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:22.185748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.186160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.187146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] PATH_INFO: `/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e/users/944109077119455a9a985402d711330c/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.213588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02ede0ee28f48a2a0314e91a1f28d0e, user_id=944109077119455a9a985402d711330c, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.217520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.346281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 971/1945] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:22 2022] PUT /identity/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e/users/944109077119455a9a985402d711330c/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:22.348243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:22.395043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9553e369b68044f1b3cc39080f4d9932'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:22.396176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9553e369b68044f1b3cc39080f4d9932'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:22.399953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:22.429981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9553e369b68044f1b3cc39080f4d9932'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:22.440106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9553e369b68044f1b3cc39080f4d9932'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:22.458784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-67fa0b24-115b-429d-bf0c-35084f7896bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9553e369b68044f1b3cc39080f4d9932'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:22.509448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865da1-66ad-4cdb-bb62-76ff3263f5be admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:22.518271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:22.519112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.519505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.519890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.520955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.522950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.544251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 972/1946] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:22.551811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:22.582759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:22.601011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 975/1947] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:22 2022] DELETE /identity/v3/users/9553e369b68044f1b3cc39080f4d9932 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:22.613597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90803ec-0c0e-4ca3-a8e5-3f76bbc53174 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:22.623235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:22.779111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:22.780866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.781627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.790032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] PATH_INFO: `/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e/users/944109077119455a9a985402d711330c/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.807937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02ede0ee28f48a2a0314e91a1f28d0e, user_id=944109077119455a9a985402d711330c, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:22.831563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:22.886266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 973/1948] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:22 2022] PUT /identity/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e/users/944109077119455a9a985402d711330c/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:22.898236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:22.898842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:22.899867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:22.964048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:22.979529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] MFA Rules not processed for user `944109077119455a9a985402d711330c`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:22.982149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:23.307720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 976/1949] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:23.734239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:23.739885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:23.789265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:23.791121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:23.867523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:23.948548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459863-c5c2-467b-9af9-95987999b969 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:23.949524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbb5b8-7739-4731-9dc9-da25c9a866de None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:23.958284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:23.959263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:23.959517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:23.959894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:23.960322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] PATH_INFO: `/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:23.961649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:23.962141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:23.962662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f41df9deab349fda6b0eb33cff0a7ae)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:23.962950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:23.965632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:23.978607 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:23.980614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c0e63-ef93-49f0-b697-3af9c224664b service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:23.995994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 974/1950] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:24.288667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f41df9deab349fda6b0eb33cff0a7ae', 'request_id': 'req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:24.306538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f41df9deab349fda6b0eb33cff0a7ae', 'request_id': 'req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:24.318780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f41df9deab349fda6b0eb33cff0a7ae', 'request_id': 'req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:24.478998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] The token cache is being invalidate because project 3f41df9deab349fda6b0eb33cff0a7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:24.479278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:24.480018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ce90f8e5-0249-4fa1-bb51-794c7a9ce20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:24.482078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 977/1951] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:23 2022] DELETE /identity/v3/projects/3f41df9deab349fda6b0eb33cff0a7ae => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:24.489068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:24.489068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:24.489502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:24.544918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:24.547419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:24.677812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:24.701627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 975/1952] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:24.719502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:24.721276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:24.721276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:24.785327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:24.787438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:24.813371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 978/1953] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:24.823379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:24.847249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f4600-7f9f-4ac6-b76f-402badc2e110 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:24.902603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:24.903552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:24.904561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:24.905076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:24.906305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:24.906973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:24.909664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:24.931444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 976/1954] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:24.938391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:24.960838 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ee178-8d6a-4753-b5fc-b1e74e334d0a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:24.983830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:24.985392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:24.986381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:24.987583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:24.991478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:24.997941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80f8bc-83b9-48a2-acb8-53f32faec68f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:25.203026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 979/1955] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:24 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:25.208874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:25.243765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eecac-6d1b-4006-9285-397df1251cfc admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:25.493863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:25.495442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.501482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.501482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.501482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:25.505041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:25.640619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.641493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.642065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.676484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 977/1956] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:25 2022] POST /identity/v3/users => generated 290 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:25.681615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:25.694388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a4349-387e-427d-95d5-2f77d10d5b7f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:25.707303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:25.708325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.708507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.708802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.709909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:25.712146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:25.733870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] MFA Rules not processed for user `4f53a28588c24f1a8940c99be7e28917`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:25.736175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:25.740799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 978/1957] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:25.748151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:25.788507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4963a-a964-4192-933e-6391f76e5e0f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:25.797310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:25.798524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.799020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.800090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.800916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:25.803936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:25.846030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 979/1958] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:25.850172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:25.868685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb743-e8e6-491b-bfec-5a1a72449159 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:25.882282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:25.883825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.884365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.884859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] PATH_INFO: `/v3/projects/fcdd807da7db4929854d7085ef2805f3/users/9749ad8d45244e439aa40ca944dedb65/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.930810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 980/1959] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:25.931483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:25.933106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:25.934960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdd807da7db4929854d7085ef2805f3, user_id=9749ad8d45244e439aa40ca944dedb65, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:25.939295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a4a7d-aa13-46fa-8dad-63686e485b16 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:25.959413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2335e-6572-4829-84fc-9d74957504f5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:25.967158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:25.968512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:25.968692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:25.968970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:25.969708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:25.971709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.074004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 980/1960] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:25 2022] PUT /identity/v3/projects/fcdd807da7db4929854d7085ef2805f3/users/9749ad8d45244e439aa40ca944dedb65/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:26.085928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.086256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.086644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.134747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] MFA Rules not processed for user `9749ad8d45244e439aa40ca944dedb65`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:26.136583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:26.171125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 981/1961] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:25 2022] POST /identity/v3/projects => generated 371 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:26.177656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.240104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 981/1962] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:26.245219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.301728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137854e-1a39-4d5e-a199-20d65faab0c0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.307356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.308045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.308453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.308732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.309452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.312177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.332398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd62cf5-d2a6-4144-b302-7716087d053c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.340265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.341418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.341917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.342458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.343780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.347630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.413983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 982/1963] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:26 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:26.417691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.419480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:26.488380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d4dc0-8d41-47fb-b6b0-d362ea754884 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.492673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.493314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.493568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.493843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.500190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 982/1964] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:26 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:26.500386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:26.500475 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.574337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0df1-81fa-4262-959b-19f24254bf96 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.577887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.579418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.579884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.581117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.581117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.581117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.581797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.583487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.590450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 983/1965] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:26.591294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.608989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 983/1966] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:26.609213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f4f6b-6b21-48d3-acef-6a10d51ef9f0 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.609213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.609397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.609655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.609878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.610630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.612599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.616357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.635105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1efa06-78c8-4e5d-bf05-007a18f40179 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.642516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.643629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.643919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.644201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] PATH_INFO: `/v3/projects/7216644e33b64343ab6f3f631731fa08/users/2f0d58b741454850acf20bedb706a5a8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.674821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7216644e33b64343ab6f3f631731fa08, user_id=2f0d58b741454850acf20bedb706a5a8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.680543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.739746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 984/1967] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:26 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:26.744025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.755550 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f248ca-1a0e-4712-af44-8c6574beffd6 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.761457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.762271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.762559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.762840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.763645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.765606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.784751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 984/1968] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:26 2022] PUT /identity/v3/projects/7216644e33b64343ab6f3f631731fa08/users/2f0d58b741454850acf20bedb706a5a8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:26.785028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 985/1969] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:26.785191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.786018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.798751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc13eb-b29b-4983-b144-ab3b6c7c1f64 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.869467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d180f75-64cd-400e-bbc7-b33279d99c2c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.878400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.879509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.880117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.880548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] PATH_INFO: `/v3/projects/f8b0dd727d96454db9a85b24f8f5b585/users/c46d64c8b9f240d1ae25d529b59d1da9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.892133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:26.904928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b0dd727d96454db9a85b24f8f5b585, user_id=c46d64c8b9f240d1ae25d529b59d1da9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.911641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.912286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.912403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.912597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.912872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.913752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.915961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.934303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 986/1970] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:26.936769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:26.948318 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839621-ad8d-4a11-80fc-cc09f983f375 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:26.953918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:26.954985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:26.955477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:26.955757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] PATH_INFO: `/v3/projects/7216644e33b64343ab6f3f631731fa08/users/2f0d58b741454850acf20bedb706a5a8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:26.958626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7216644e33b64343ab6f3f631731fa08, user_id=2f0d58b741454850acf20bedb706a5a8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:26.961406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:26.980060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 987/1971] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:26 2022] PUT /identity/v3/projects/7216644e33b64343ab6f3f631731fa08/users/2f0d58b741454850acf20bedb706a5a8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:27.052395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:27.109031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 985/1972] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:26 2022] PUT /identity/v3/projects/f8b0dd727d96454db9a85b24f8f5b585/users/c46d64c8b9f240d1ae25d529b59d1da9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:27.127975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:27.209721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac852625-42a0-427f-99a4-16c10e4c1402 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:27.282484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:27.283532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:27.283872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:27.284393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:27.285242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:27.287288 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93888a15-38a4-4ce8-abab-26a06ff485e8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:27.302148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 988/1973] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:27.303640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:27.315227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417d26-295b-4218-82b1-3e57455db156 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:27.320044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:27.320957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:27.321241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:27.321518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] PATH_INFO: `/v3/projects/f8b0dd727d96454db9a85b24f8f5b585/users/c46d64c8b9f240d1ae25d529b59d1da9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:27.323694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b0dd727d96454db9a85b24f8f5b585, user_id=c46d64c8b9f240d1ae25d529b59d1da9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:27.326434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:27.339795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 986/1974] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:27 2022] PUT /identity/v3/projects/f8b0dd727d96454db9a85b24f8f5b585/users/c46d64c8b9f240d1ae25d529b59d1da9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:27.351082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:27.351425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:27.351763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:27.382271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] MFA Rules not processed for user `c46d64c8b9f240d1ae25d529b59d1da9`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:27.383534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:27.604168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 989/1975] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:28.031757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:28.105528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4fe8-5a61-45fb-ab35-ff5fa72962e5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:28.110428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:28.114282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:28.114282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:28.114282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:28.123792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:28.125400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302d72a-35ea-49ad-8095-51d556f949d2 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:28.153918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 987/1976] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:28.240797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:29.134817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:29.245473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8557ad-1463-4ad2-9bfc-784746efcb15 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:29.252862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:29.254018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:29.254363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:29.254636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] PATH_INFO: `/v3/users/9749ad8d45244e439aa40ca944dedb65` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:29.257783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9749ad8d45244e439aa40ca944dedb65)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:29.259942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:29.463284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:29.465583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:29.465583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:29.505780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] MFA Rules not processed for user `2f0d58b741454850acf20bedb706a5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:29.507097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:29.558913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9749ad8d45244e439aa40ca944dedb65'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:29.559924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9749ad8d45244e439aa40ca944dedb65'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:29.584965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9749ad8d45244e439aa40ca944dedb65'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:29.593362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9749ad8d45244e439aa40ca944dedb65'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:29.610709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85d4d238-c4e1-4c93-86a8-a6ecb12b19b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9749ad8d45244e439aa40ca944dedb65'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:29.616090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 988/1977] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:29.674706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:29.678897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:29.692674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 990/1978] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:29 2022] DELETE /identity/v3/users/9749ad8d45244e439aa40ca944dedb65 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:29.703303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824f8b-f4ab-4b6d-b3e7-b2d90254d2ff None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:29.705504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:29.705801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:29.706076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:29.729596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:29.739130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:29.816655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:29.817909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:29.818187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:29.829605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:29.829605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:29.840807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 991/1979] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:29.856769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:29.859265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:29.916462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:29.923019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:29.928277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:29.944560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06fc7f-0f6e-4d74-a154-f5477f94d5c1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:29.946014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 989/1980] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:29.952757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:29.953679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:29.956010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:29.956010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:29.971692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:29.974501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:29.993149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 992/1981] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:30.917038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:30.982659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:30.984681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:31.000174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53558-cb51-43e0-bd7c-c9d952e24d02 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:31.004819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:31.005724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:31.006002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:31.006310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] PATH_INFO: `/v3/projects/fcdd807da7db4929854d7085ef2805f3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:31.008116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdd807da7db4929854d7085ef2805f3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:31.011597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:31.193212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdd807da7db4929854d7085ef2805f3', 'request_id': 'req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.200341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdd807da7db4929854d7085ef2805f3', 'request_id': 'req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.206373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdd807da7db4929854d7085ef2805f3', 'request_id': 'req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.378133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] The token cache is being invalidate because project fcdd807da7db4929854d7085ef2805f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:31.378457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.386842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.386842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 990/1982] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:30 2022] DELETE /identity/v3/projects/fcdd807da7db4929854d7085ef2805f3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:31.389739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:31.400026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e784-c954-436d-9f5c-ecb8dd082900 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:31.475621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:31.476528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:31.476831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:31.477084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] PATH_INFO: `/v3/users/c46d64c8b9f240d1ae25d529b59d1da9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:31.478498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46d64c8b9f240d1ae25d529b59d1da9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:31.480459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:31.756773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46d64c8b9f240d1ae25d529b59d1da9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.757599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46d64c8b9f240d1ae25d529b59d1da9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.774975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46d64c8b9f240d1ae25d529b59d1da9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.775785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46d64c8b9f240d1ae25d529b59d1da9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.786422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-21f6f4c7-5d9e-4efb-b78e-7769d1f08189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46d64c8b9f240d1ae25d529b59d1da9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:31.887990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 993/1983] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:31 2022] DELETE /identity/v3/users/c46d64c8b9f240d1ae25d529b59d1da9 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:32.137916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:32.245124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:32.631070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:32.632523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:32.646871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b8a21-d108-4510-bbfd-dfc601c8f82d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:32.717180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:32.718101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:32.718438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:32.718714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] PATH_INFO: `/v3/projects/f8b0dd727d96454db9a85b24f8f5b585` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:32.720575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b0dd727d96454db9a85b24f8f5b585)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:32.722696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:32.903495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b0dd727d96454db9a85b24f8f5b585', 'request_id': 'req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:32.911803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b0dd727d96454db9a85b24f8f5b585', 'request_id': 'req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:32.918898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b0dd727d96454db9a85b24f8f5b585', 'request_id': 'req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:33.011855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] The token cache is being invalidate because project f8b0dd727d96454db9a85b24f8f5b585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:33.012517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:33.013362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e6549b7c-6b0f-4c1c-8258-225cf119a6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:33.015934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 991/1984] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:32 2022] DELETE /identity/v3/projects/f8b0dd727d96454db9a85b24f8f5b585 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:33.023625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.024440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.025118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.053862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:33.055392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:33.138128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 994/1985] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:33.148023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.148508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.148976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.181048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:33.183108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:33.199852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 992/1986] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:33.203607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.215110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24dc43-8983-4c89-8750-591c6ff8762f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.220659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.221336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.221637 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.221910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.222679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:33.222985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.224620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.239678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 995/1987] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:33.243776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.255227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9555b0a-9cf8-4c51-807f-6aef5eff9fe5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.262787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.265425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.265921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.266523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.267858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.271024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.388156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 993/1988] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:33 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:33.394320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.409319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fcc4-e9f1-456d-acfb-e62029210fc3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.508418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.509010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.509286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.509559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.510299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.512303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.598246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 996/1989] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:33 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:33.605478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.611312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.613417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:33.629086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9b4ac-c041-45a6-b7d6-0df5392ff5b6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.637922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.639947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.640666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.641156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.642384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.645665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.671785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 994/1990] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:33.674031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.694016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20095aa-db0a-4ecc-b257-a9f7503c66b0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.703194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.705276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.705847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.706348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.707767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.711145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.714120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969114dc-03e3-4bca-b491-95d791c115a9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.719233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.721093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.721093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.721093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.723892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:33.731307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 995/1991] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:33.736510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.748320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d5566-1ddc-47c7-8b97-574a4c419afd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:33.755210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:33.756284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.756694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.757391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] PATH_INFO: `/v3/projects/c2b497725c534066b011d09c3241b13d/users/70e110edb9a4443cb08dd9e1e13d07aa/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.777736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b497725c534066b011d09c3241b13d, user_id=70e110edb9a4443cb08dd9e1e13d07aa, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.781303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.821434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:33.822971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628f151-4f61-4d6b-8308-a7d6b8508549 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:33.841404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 997/1992] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:33.886876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 996/1993] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:33 2022] PUT /identity/v3/projects/c2b497725c534066b011d09c3241b13d/users/70e110edb9a4443cb08dd9e1e13d07aa/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:33.907570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:33.908091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:33.908754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:33.916338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:33.945681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:33.947926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:33.968628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] MFA Rules not processed for user `70e110edb9a4443cb08dd9e1e13d07aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:33.971818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:34.038009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e2ac6-f97c-456b-9ba8-3cd2a55994eb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:34.048172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:34.049209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:34.049209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:34.049727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:34.051875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:34.095148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 998/1994] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:34.099056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:34.103833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:34.181676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:34.185143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:34.191832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a031e3-f9f2-431b-93a1-2c069ac8551a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:34.198492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:34.199214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:34.199748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:34.200065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:34.200776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:34.203408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:34.204125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 997/1995] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:34.452485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 999/1996] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:34 2022] POST /identity/v3/projects => generated 381 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:34.456613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:34.475224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1847-eb73-4fab-947c-a0bdc3a30d75 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:34.546989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:34.547884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:34.548289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:34.548689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:34.549649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:34.552261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:34.743019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 998/1997] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:34 2022] POST /identity/v3/users => generated 297 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:34.743337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:34.755464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e41c65-f31e-4dd0-a940-32c5fb3272a1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:34.763342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:34.764726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:34.765319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:34.765725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:34.767178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:34.769672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:34.791683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1000/1998] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:34.796289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:34.807871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaa04b-f4f7-4eeb-93e2-4fd9122085f7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:34.814133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:34.821807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:34.821807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:34.821807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] PATH_INFO: `/v3/projects/305e51a4e0d647ab82f061c8a596c4d5/users/2834cab41898416bbd7a73ed79890beb/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:34.834405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305e51a4e0d647ab82f061c8a596c4d5, user_id=2834cab41898416bbd7a73ed79890beb, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:34.837119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:34.959016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 999/1999] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:34 2022] PUT /identity/v3/projects/305e51a4e0d647ab82f061c8a596c4d5/users/2834cab41898416bbd7a73ed79890beb/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:34.963522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:34.974916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d209f-3fd5-4fb1-b837-3062d7453621 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:35.047322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:35.047710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:35.047991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:35.048262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:35.049033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:35.050794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f348f6-357e-4f41-87ed-8f7a8cb61088 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:35.063506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1001/2000] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:35.067930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:35.079731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bed9f-abd0-47fb-87f7-5a5d10746d74 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:35.085748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:35.087086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:35.091897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:35.091897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] PATH_INFO: `/v3/projects/305e51a4e0d647ab82f061c8a596c4d5/users/2834cab41898416bbd7a73ed79890beb/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:35.091897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=305e51a4e0d647ab82f061c8a596c4d5, user_id=2834cab41898416bbd7a73ed79890beb, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:35.094299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:35.106994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1000/2001] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:35 2022] PUT /identity/v3/projects/305e51a4e0d647ab82f061c8a596c4d5/users/2834cab41898416bbd7a73ed79890beb/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:35.118863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:35.119146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:35.119470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:35.149694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] MFA Rules not processed for user `2834cab41898416bbd7a73ed79890beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:35.151056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:35.230742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:35.274652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1002/2002] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:35.639995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:35.860862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:35.943355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ffc3-86a8-4fa1-acbb-eee3ca68e51f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:35.948060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:35.948736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:35.949012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:35.949353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:35.960372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:35.962389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:35.977397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1001/2003] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:36.405015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:36.405649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:36.422968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfa2ea-6cd3-45b3-9797-e099e182e60d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:36.427786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:36.428589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:36.428995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:36.429340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:36.509294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:36.512183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03147d38-44f2-404b-b69e-b3792383a86f service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:36.529396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1003/2004] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:36.781203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:37.581848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:37.675296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511a408-32fd-414d-9860-8ba2a5008cd6 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:37.681306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:37.682375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:37.682648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:37.682923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] PATH_INFO: `/v3/users/70e110edb9a4443cb08dd9e1e13d07aa` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:37.684174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e110edb9a4443cb08dd9e1e13d07aa)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:37.686231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:37.806981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:37.896594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:37.995492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e110edb9a4443cb08dd9e1e13d07aa'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:38.002794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e110edb9a4443cb08dd9e1e13d07aa'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:38.016037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e110edb9a4443cb08dd9e1e13d07aa'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:38.016712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e110edb9a4443cb08dd9e1e13d07aa'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:38.146372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e110edb9a4443cb08dd9e1e13d07aa'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:38.153504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1002/2005] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:37 2022] DELETE /identity/v3/users/70e110edb9a4443cb08dd9e1e13d07aa => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:38.159133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae2f4-c643-460b-9726-0331106e76be None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:38.159325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae2f4-c643-460b-9726-0331106e76be None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:38.159648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae2f4-c643-460b-9726-0331106e76be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:38.193284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-038ae2f4-c643-460b-9726-0331106e76be None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:38.194624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-038ae2f4-c643-460b-9726-0331106e76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:38.302240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:38.315683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1004/2006] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:38.337385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:38.339885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:38.473185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce35f2-30bb-4dd7-bb55-ea06daf84462 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:38.478890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:38.479903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:38.480335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:38.480792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:38.493915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:38.495519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f31603-be2a-4174-b90f-ac1657eced0e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:38.527370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1003/2007] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:38.581045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:39.228049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:39.356897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ae2f4-c643-460b-9726-0331106e76be None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:39.358713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-038ae2f4-c643-460b-9726-0331106e76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.387812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ae2f4-c643-460b-9726-0331106e76be None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:39.398234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:39.399403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.399880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.400317 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] PATH_INFO: `/v3/projects/c2b497725c534066b011d09c3241b13d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.401883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b497725c534066b011d09c3241b13d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:39.404588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:39.512342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdb00-c8b4-46c2-9e78-0af261a56836 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.512626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdb00-c8b4-46c2-9e78-0af261a56836 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.512775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdb00-c8b4-46c2-9e78-0af261a56836 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.515765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1004/2008] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:37:39.525422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.526009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.526323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.564156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] MFA Rules not processed for user `dc3407db3d994b17851618218efcb7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:39.566594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.644508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b497725c534066b011d09c3241b13d', 'request_id': 'req-31468604-817c-4e94-be62-14c6380d89fe'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:39.652135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b497725c534066b011d09c3241b13d', 'request_id': 'req-31468604-817c-4e94-be62-14c6380d89fe'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:39.659356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b497725c534066b011d09c3241b13d', 'request_id': 'req-31468604-817c-4e94-be62-14c6380d89fe'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:39.683212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1005/2009] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:39.705140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:39.707506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.735762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d8b1f-bca4-4efb-a0ba-1f27532f5c5e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:39.757051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] The token cache is being invalidate because project c2b497725c534066b011d09c3241b13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:39.757361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:39.758433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:39.761391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1005/2010] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:39 2022] DELETE /identity/v3/projects/c2b497725c534066b011d09c3241b13d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:39.762275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:39.763966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.828819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:39.829830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.830293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.831033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.834311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.849449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:39.853283 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:39.865133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468604-817c-4e94-be62-14c6380d89fe admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:39.872306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:39.873325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1006/2011] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:39.874513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.875000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.875456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] PATH_INFO: `/v3/users/2ca8c6589d0a4bbb9c5adf780cbd2fd8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.875897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:39.877816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca8c6589d0a4bbb9c5adf780cbd2fd8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:39.878644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:39.881529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:39.902711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15efe-4af6-4b9a-8b89-73be4bcf5a11 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:39.913206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:39.915026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:39.915691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:39.916258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] PATH_INFO: `/v3/users/2834cab41898416bbd7a73ed79890beb` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:39.918772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2834cab41898416bbd7a73ed79890beb)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:39.923015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:40.146856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca8c6589d0a4bbb9c5adf780cbd2fd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.146856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca8c6589d0a4bbb9c5adf780cbd2fd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.150342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2834cab41898416bbd7a73ed79890beb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.160140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2834cab41898416bbd7a73ed79890beb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.164602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca8c6589d0a4bbb9c5adf780cbd2fd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.165611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca8c6589d0a4bbb9c5adf780cbd2fd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.179176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca8c6589d0a4bbb9c5adf780cbd2fd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.181162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2834cab41898416bbd7a73ed79890beb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.182483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2834cab41898416bbd7a73ed79890beb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.285236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1006/2012] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:39 2022] DELETE /identity/v3/users/2ca8c6589d0a4bbb9c5adf780cbd2fd8 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:40.337734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2834cab41898416bbd7a73ed79890beb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.349298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1007/2013] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:39 2022] DELETE /identity/v3/users/2834cab41898416bbd7a73ed79890beb => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:40.384507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:40.478549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:40.478549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:40.566676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97252292-5270-481d-ac53-8d5fac705b87 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:40.646815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:40.648427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:40.649011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:40.649542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] PATH_INFO: `/v3/users/ed77f93c28db4b27b5d0e29ba4b208eb` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:40.652618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed77f93c28db4b27b5d0e29ba4b208eb)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:40.661328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:40.669131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:40.670954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:40.705087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc72420-eb0a-4066-9af6-ad2da4a2904a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:40.714515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:40.716289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:40.716433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:40.716433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] PATH_INFO: `/v3/projects/56f6fbec499344859b9da684d29ffdb8` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:40.719829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f6fbec499344859b9da684d29ffdb8)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:40.723628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:40.941805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed77f93c28db4b27b5d0e29ba4b208eb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.943208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed77f93c28db4b27b5d0e29ba4b208eb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.951159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f6fbec499344859b9da684d29ffdb8', 'request_id': 'req-c9d95ff9-c925-4655-b974-52bb80f71ef4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.957814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed77f93c28db4b27b5d0e29ba4b208eb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.958647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed77f93c28db4b27b5d0e29ba4b208eb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.960291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f6fbec499344859b9da684d29ffdb8', 'request_id': 'req-c9d95ff9-c925-4655-b974-52bb80f71ef4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.966805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3622220-5174-48a8-9389-045b931fb62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed77f93c28db4b27b5d0e29ba4b208eb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:40.967035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f6fbec499344859b9da684d29ffdb8', 'request_id': 'req-c9d95ff9-c925-4655-b974-52bb80f71ef4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.051946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1007/2014] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:40 2022] DELETE /identity/v3/users/ed77f93c28db4b27b5d0e29ba4b208eb => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:41.055538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:41.058769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.059110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.059335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.079050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:41.080211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.142887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1008/2015] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:41.152758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.154548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.161369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] The token cache is being invalidate because project 56f6fbec499344859b9da684d29ffdb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:41.162157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.163649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c9d95ff9-c925-4655-b974-52bb80f71ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.166956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1008/2016] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:40 2022] DELETE /identity/v3/projects/56f6fbec499344859b9da684d29ffdb8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:41.174749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.175217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.175702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.214692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:41.216393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.226025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1009/2017] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:41.227268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.228788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.242867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f864a3-a767-4038-9c57-619132ed4021 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.250769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.250769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.250769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.250769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] PATH_INFO: `/v3/projects/305e51a4e0d647ab82f061c8a596c4d5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.251348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305e51a4e0d647ab82f061c8a596c4d5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.252747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.254577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5663b8-a405-4570-a55a-a34ae8dda87c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.258632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.259301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.260133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.260133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.261791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.275243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.276692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6a941-c91e-4574-9869-bd0c2fef64be service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.294526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1009/2018] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:41.297283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.297571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.297851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.319664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:41.321122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.332564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1010/2019] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:41.337022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.347161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac9fdf-9cb4-49f2-bb1d-eb3f862814b5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.352823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.353777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.354061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.354373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.355118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:41.355425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.357118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.372141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1011/2020] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:37:41.376837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.386937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0cf-e542-4dc8-89ce-7f84af50f76b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.392748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.393473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.393701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.393977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.394892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.396932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.488700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305e51a4e0d647ab82f061c8a596c4d5', 'request_id': 'req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.495046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305e51a4e0d647ab82f061c8a596c4d5', 'request_id': 'req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.501199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305e51a4e0d647ab82f061c8a596c4d5', 'request_id': 'req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.619774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] The token cache is being invalidate because project 305e51a4e0d647ab82f061c8a596c4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:41.622844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.623012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1012/2021] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:41.623172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1eae7d59-32f1-41e4-9ca1-aa8c1cf20b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:41.624336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1010/2022] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:41 2022] DELETE /identity/v3/projects/305e51a4e0d647ab82f061c8a596c4d5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:41.625977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.628366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.630581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccc58-bf86-4daa-be69-6396d4f7af7c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.631413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccc58-bf86-4daa-be69-6396d4f7af7c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.631913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccc58-bf86-4daa-be69-6396d4f7af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.665181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-084ccc58-bf86-4daa-be69-6396d4f7af7c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:41.666586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-084ccc58-bf86-4daa-be69-6396d4f7af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.711773 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f6b4-9b16-4298-a129-01bd98c31f43 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.718450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.719136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.719414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.719691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.720411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.722457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.754174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1011/2023] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:41.769191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.769491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.769745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.789656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:41.790892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:41.799883 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1012/2024] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:41.806241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.818925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb7ce-ec6b-4d3e-8cd8-42adacb3466e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.824664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.825652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.826075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.826495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.828151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:41.828614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.830525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.847048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1013/2025] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:37:41.854019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1013/2026] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:41.854261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.858591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.866054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede5cb-f8ac-4247-b6ca-6976d1cd26f4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.872941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f05ff-a4ea-4de2-83ec-330f86bda933 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.873219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.874464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.874752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.875033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.875750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.877734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.879983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.880808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.881216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.881618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.882549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.884595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.897036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1014/2027] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:41.903331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.913986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8451b-7ce0-469a-8550-b701c3eb1867 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.919919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.920103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.920395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.920671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.921406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:41.923129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:41.937702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1015/2028] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:37:41.943275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:41.959147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322d7d-1cef-4c2f-9de4-e3c3de450a7c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:41.968984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:41.970883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:41.971341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:41.971808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] PATH_INFO: `/v3/projects/91f6b3c676a84127989cc8e35ce91de1/users/5132dcb5580d44e1bc2c7cf4a6893678/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:41.998004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f6b3c676a84127989cc8e35ce91de1, user_id=5132dcb5580d44e1bc2c7cf4a6893678, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.001075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.027455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1014/2029] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:41 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:42.028644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.030974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.097011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:42.103070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1016/2030] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:41 2022] PUT /identity/v3/projects/91f6b3c676a84127989cc8e35ce91de1/users/5132dcb5580d44e1bc2c7cf4a6893678/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:42.105337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ab68-808e-4699-a83a-9080d421b63f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.107663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.180608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.181544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.181829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.182100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] PATH_INFO: `/v3/projects/2349a41e9b2544aea5d31365db5ad521` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.183400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2349a41e9b2544aea5d31365db5ad521)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.185341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.189888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95442f-7cc5-4b5d-929c-af62e6bc791d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.190770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:42.195062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.195745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.196017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.200126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.200126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.200126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29087b4-e808-44d5-9899-dfa31b2bf56a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.300911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1017/2031] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:42 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:42.303386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.303896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.304619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.330537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2349a41e9b2544aea5d31365db5ad521', 'request_id': 'req-2735ac84-1700-4e4d-87c3-bc3bf07af256'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.338583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2349a41e9b2544aea5d31365db5ad521', 'request_id': 'req-2735ac84-1700-4e4d-87c3-bc3bf07af256'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.345317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2349a41e9b2544aea5d31365db5ad521', 'request_id': 'req-2735ac84-1700-4e4d-87c3-bc3bf07af256'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.367839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:42.369892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.434092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1018/2032] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:42.434911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.446949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6638a-6b51-40b5-bbfc-8fed2aef5a92 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.452045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.452993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.452993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.453289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.454052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.455824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.468481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1019/2033] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:42.469438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.471521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.533556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fe14e-1f08-48e3-bbb6-9f26665ecb61 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.535608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] The token cache is being invalidate because project 2349a41e9b2544aea5d31365db5ad521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:42.536138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.536681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.537409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.538051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.538371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.538681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.541577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1015/2034] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:42 2022] DELETE /identity/v3/projects/2349a41e9b2544aea5d31365db5ad521 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:42.542564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.543910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.544879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.557939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.559435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.563919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ac84-1700-4e4d-87c3-bc3bf07af256 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.571421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1020/2035] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:42.572309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.573742 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.575531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.576225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.576810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.577107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.577911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.579850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.586340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b42c-29e2-4943-a5ff-08f8187777dc service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.591289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.592767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.593046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.593314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] PATH_INFO: `/v3/users/b0617d4fbbfb4ae0ad3359f4872450c4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.594674 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0617d4fbbfb4ae0ad3359f4872450c4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.595682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1016/2036] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:42.596864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.602036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.617736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507fe06-f618-40a5-a7a4-c28cb57a3cb8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:42.623126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:42.624063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.624344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.624621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] PATH_INFO: `/v3/projects/7cfedfba7b154c0999c294601945d813/users/8c12eb4d4c924e2ab0350249fe35d0da/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.641599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cfedfba7b154c0999c294601945d813, user_id=8c12eb4d4c924e2ab0350249fe35d0da, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:42.644534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124dba0-68ef-4f17-a095-ecbed0c7331b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:42.759741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1017/2037] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:42 2022] PUT /identity/v3/projects/7cfedfba7b154c0999c294601945d813/users/8c12eb4d4c924e2ab0350249fe35d0da/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:42.771673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:42.771991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:42.772200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:42.814278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] MFA Rules not processed for user `8c12eb4d4c924e2ab0350249fe35d0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:42.816387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.909145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1018/2038] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:42.920801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:42.922573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:42.938953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0617d4fbbfb4ae0ad3359f4872450c4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.939651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0617d4fbbfb4ae0ad3359f4872450c4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.954008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0617d4fbbfb4ae0ad3359f4872450c4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.955113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0617d4fbbfb4ae0ad3359f4872450c4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:42.965552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0617d4fbbfb4ae0ad3359f4872450c4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:43.005320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fca43-b1e0-44d1-8166-adcd0a2fb1f8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:43.005320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:43.005320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:43.005320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:43.005930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:43.007061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:43.020318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:43.021782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:43.032681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1019/2039] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:43.106262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1021/2040] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:42 2022] DELETE /identity/v3/users/b0617d4fbbfb4ae0ad3359f4872450c4 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:43.129319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:43.281082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:43.659882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:43.661851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:43.766858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427275-08a1-40ba-a777-443d29efff1d service glance] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:43.841153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:43.842843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:43.843130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:43.843404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] PATH_INFO: `/v3/projects/dae2e3c472204e979f233ae454dda436` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:43.844689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae2e3c472204e979f233ae454dda436)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:43.846669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:43.975054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae2e3c472204e979f233ae454dda436', 'request_id': 'req-888908b9-2ef9-42ab-89e0-a0d512140bac'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:43.984285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae2e3c472204e979f233ae454dda436', 'request_id': 'req-888908b9-2ef9-42ab-89e0-a0d512140bac'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:43.992543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae2e3c472204e979f233ae454dda436', 'request_id': 'req-888908b9-2ef9-42ab-89e0-a0d512140bac'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.084944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] The token cache is being invalidate because project dae2e3c472204e979f233ae454dda436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:44.084944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.085605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.090465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1020/2041] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:43 2022] DELETE /identity/v3/projects/dae2e3c472204e979f233ae454dda436 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:44.091267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:44.105851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8346-de7b-4f80-8779-0decf978cbc0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:44.192695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:44.210441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:44.210995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:44.211407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] PATH_INFO: `/v3/users/944109077119455a9a985402d711330c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:44.217720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944109077119455a9a985402d711330c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:44.219658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:44.290990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:44.635809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944109077119455a9a985402d711330c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.637367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944109077119455a9a985402d711330c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.654815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944109077119455a9a985402d711330c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.655964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944109077119455a9a985402d711330c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.665610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ee28be92-476a-4117-a22f-dd10c68d5b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944109077119455a9a985402d711330c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:44.760680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1022/2042] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:44 2022] DELETE /identity/v3/users/944109077119455a9a985402d711330c => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:44.856703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:45.378277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:45.505237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:45.505237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:45.525373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888908b9-2ef9-42ab-89e0-a0d512140bac admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:45.642930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:45.648322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:45.649054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:45.649771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] PATH_INFO: `/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:45.652489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02ede0ee28f48a2a0314e91a1f28d0e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:45.658004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:45.798112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02ede0ee28f48a2a0314e91a1f28d0e', 'request_id': 'req-736c4f56-733e-4fd6-b3d3-8455ea141939'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:45.805674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02ede0ee28f48a2a0314e91a1f28d0e', 'request_id': 'req-736c4f56-733e-4fd6-b3d3-8455ea141939'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:45.816212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02ede0ee28f48a2a0314e91a1f28d0e', 'request_id': 'req-736c4f56-733e-4fd6-b3d3-8455ea141939'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:45.897434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] The token cache is being invalidate because project e02ede0ee28f48a2a0314e91a1f28d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:45.898070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:45.898356 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-736c4f56-733e-4fd6-b3d3-8455ea141939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:45.901892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1021/2043] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:45 2022] DELETE /identity/v3/projects/e02ede0ee28f48a2a0314e91a1f28d0e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:45.906213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:45.908825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:45.908825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:45.926508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:45.927739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:45.994107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1023/2044] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:46.005342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.005661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.005892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.046778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:46.048146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:46.064314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1022/2045] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:46.065734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.077419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc747b-7194-411c-af2d-b4d8690dd96d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.083375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.084460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.084927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.090688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.093524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:46.094242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.098184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.134989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1024/2046] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:46.135952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.152543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88853d68-5a73-4ab3-b994-57f07fad482c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.160957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.162175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.162789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.163371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.164266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.166669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.310599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1023/2047] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:46 2022] POST /identity/v3/projects => generated 391 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:46.316735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.334469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8b1c9-5ad3-441c-876d-f1517ad33aef admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.353764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:46.480382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.483250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.483250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.483250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.483250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.485308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.618287 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1025/2048] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:46 2022] POST /identity/v3/users => generated 296 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:46.627840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.639617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d47b06-dfe0-4058-a065-8d34c37fd3b7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.653491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.654728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.657835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.658407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.660088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.662746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.677432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1024/2049] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:46.687460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.696642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5871ec1-2538-49ee-81fd-aa4ecbaa66e8 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.701883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.702958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.703316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.703668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.704764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.706525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192cb66-a8e2-4ac0-b7cd-6e1c91b07750 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.719337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1026/2050] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:46.727793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:46.739920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cc46e-aed6-40b4-a643-95b99ba8dc8f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:46.751741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:46.752874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.753453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.754248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] PATH_INFO: `/v3/projects/7055e2f1298f4b2386204d12c6d7871f/users/e301ec757c2344779a0d73f0136af8e1/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.776406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7055e2f1298f4b2386204d12c6d7871f, user_id=e301ec757c2344779a0d73f0136af8e1, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:46.780148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:46.910621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1025/2051] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:46 2022] PUT /identity/v3/projects/7055e2f1298f4b2386204d12c6d7871f/users/e301ec757c2344779a0d73f0136af8e1/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:46.925267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:46.925988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:46.926722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:46.971997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] MFA Rules not processed for user `e301ec757c2344779a0d73f0136af8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:46.973237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:47.062611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1027/2052] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:47.067470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:47.168466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee630c4-9ba1-40d9-87f4-e04cd1fc26d6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:47.177464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:47.178902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:47.179776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:47.180350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:47.182038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:47.186249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:47.358162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1026/2053] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:47 2022] POST /identity/v3/projects => generated 393 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:47.359562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:47.371673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829ae7a-0a17-4598-935b-58913292f638 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:47.490248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:47.491091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:47.491432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:47.491708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:47.492797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:47.495215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:47.507012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:47.625951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1028/2054] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:47 2022] POST /identity/v3/users => generated 303 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:47.631906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:47.651645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f18e7-a5d6-4a08-80d7-55d8dbd25f98 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:47.660918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:47.663057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:47.663776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:47.664451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:47.666153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:47.669627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:47.688293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:47.704952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:47.714234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1027/2055] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:47.724427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b0577-aff0-42ad-b249-98531e86c857 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:47.730563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:47.732192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:47.732749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:47.733759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] PATH_INFO: `/v3/projects/309ed3708078464c8a97dabbe48b6cf9/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:47.764620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] RBAC: Authorizing `identity:create_grant(project_id=309ed3708078464c8a97dabbe48b6cf9, user_id=cd3f1b3c9d354bcfa2a941e5f1ffbe5d, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:47.770106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:47.897515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1029/2056] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:47 2022] PUT /identity/v3/projects/309ed3708078464c8a97dabbe48b6cf9/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:47.902183 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:47.914190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6b24a-7715-42be-885e-35942c25183a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:47.995611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:47.996655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:47.997122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:47.997540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:47.998625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:48.000448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997a4d-f6e8-435d-ae1f-0443cd77048e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:48.015370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1028/2057] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:37:48.017126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:48.028819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7f87-4660-4dcc-9e9e-d2d8107c017c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:48.035156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:48.036472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:48.037308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:48.037308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] PATH_INFO: `/v3/projects/309ed3708078464c8a97dabbe48b6cf9/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:48.039460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309ed3708078464c8a97dabbe48b6cf9, user_id=cd3f1b3c9d354bcfa2a941e5f1ffbe5d, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:48.042614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:48.056139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1030/2058] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:37:48 2022] PUT /identity/v3/projects/309ed3708078464c8a97dabbe48b6cf9/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:48.067915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:48.067915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:48.067915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:48.098907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] MFA Rules not processed for user `cd3f1b3c9d354bcfa2a941e5f1ffbe5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:48.100373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:48.190471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1029/2059] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:48.205410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:48.207904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:48.303357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690662-6e02-4402-b9c8-83bd6e311fd2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:48.312337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:48.313223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:48.313617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:48.313963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:48.326962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:48.329021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3d131-aa50-4336-a58d-035d5f0604d9 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:48.354023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1031/2060] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:48.464764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:48.466355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:48.495426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbcae9-a7d7-4c08-a604-f8a7be59170a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:48.502732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:48.503589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:48.503997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:48.504385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:48.603242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:48.653083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:48.655661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0963a8-48bb-44e6-8728-9c60a08339ea service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:48.684063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1030/2061] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:49.458854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:50.304970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:50.305474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:50.305906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:50.349077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] MFA Rules not processed for user `5132dcb5580d44e1bc2c7cf4a6893678`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:50.351002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:50.505052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1032/2062] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:50.509846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080d93-fdef-4ef1-b4d8-8781c7ee580f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:50.510728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080d93-fdef-4ef1-b4d8-8781c7ee580f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:50.517164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080d93-fdef-4ef1-b4d8-8781c7ee580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:50.541461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-17080d93-fdef-4ef1-b4d8-8781c7ee580f None None] MFA Rules not processed for user `5132dcb5580d44e1bc2c7cf4a6893678`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:50.544298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-17080d93-fdef-4ef1-b4d8-8781c7ee580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:50.563539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1031/2063] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:50.581969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:50.584200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:50.698058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:50.705291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d55c-3287-4afc-8c37-66463ee22ddb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:50.711747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:50.712560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:50.712843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:50.713126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:50.724359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:50.726148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:50.741293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1033/2064] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:50.823066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:51.514416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:51.722845 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbcef8-0cec-491c-9781-95a10f719c2b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:51.724965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbcef8-0cec-491c-9781-95a10f719c2b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:51.724965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbcef8-0cec-491c-9781-95a10f719c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:51.758768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fcdbcef8-0cec-491c-9781-95a10f719c2b None None] MFA Rules not processed for user `5132dcb5580d44e1bc2c7cf4a6893678`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:51.760724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fcdbcef8-0cec-491c-9781-95a10f719c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:51.772292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1032/2065] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:52.612513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:53.351905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:53.813696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:53.819839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:54.331571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:54.344851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f2609-baae-416a-b07c-f1f9f1ca231a service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:54.363760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:54.366466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:54.367168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:54.367798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] PATH_INFO: `/v3/users/91bbd7e313394e04b2aaf81dd91612c9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:54.371608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bbd7e313394e04b2aaf81dd91612c9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:54.381410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:54.777906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bbd7e313394e04b2aaf81dd91612c9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:54.778793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bbd7e313394e04b2aaf81dd91612c9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:54.804481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bbd7e313394e04b2aaf81dd91612c9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:54.805236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bbd7e313394e04b2aaf81dd91612c9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:54.815647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bbd7e313394e04b2aaf81dd91612c9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:54.942607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1034/2066] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:53 2022] DELETE /identity/v3/users/91bbd7e313394e04b2aaf81dd91612c9 => generated 0 bytes in 1130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:54.959735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:54.960300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:54.961496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:54.993072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:54.994690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:55.102882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:55.106605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1033/2067] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:55.123510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:55.125898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:55.295826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400ec2f-732e-4ac2-b60e-acd967525a4d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:55.302611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:55.303853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:55.304620 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:55.305376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:55.323142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:55.326111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:55.347018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1035/2068] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:37:56.204563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:56.206467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:56.235803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff47b-6977-4710-a3b7-e6d7a9358204 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:56.249869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:56.254597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:56.254597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:56.254597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] PATH_INFO: `/v3/projects/a9d76992788b4e108127bb3a5177fa30` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:56.255095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d76992788b4e108127bb3a5177fa30)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:56.257797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:56.490675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:56.571239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d76992788b4e108127bb3a5177fa30', 'request_id': 'req-28bef743-329a-4e7b-bd13-40aa10d8920e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:56.597504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d76992788b4e108127bb3a5177fa30', 'request_id': 'req-28bef743-329a-4e7b-bd13-40aa10d8920e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:56.605605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d76992788b4e108127bb3a5177fa30', 'request_id': 'req-28bef743-329a-4e7b-bd13-40aa10d8920e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:56.709019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] The token cache is being invalidate because project a9d76992788b4e108127bb3a5177fa30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:56.710100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:56.711834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:56.715032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1034/2069] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:56 2022] DELETE /identity/v3/projects/a9d76992788b4e108127bb3a5177fa30 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:56.720211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:56.755489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247eb7-578e-4c8c-8dab-be68aca7fdc5 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:56.861944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:56.865067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:56.868022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:56.868571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] PATH_INFO: `/v3/users/0d514aeb7ce148f189d167b1dced14dc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:56.872375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d514aeb7ce148f189d167b1dced14dc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:56.878399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:57.067640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:57.068160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:57.084731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef743-329a-4e7b-bd13-40aa10d8920e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:57.089708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:57.090652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:57.090936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:57.091216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] PATH_INFO: `/v3/users/e301ec757c2344779a0d73f0136af8e1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:57.092597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e301ec757c2344779a0d73f0136af8e1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:57.094628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:57.152513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d514aeb7ce148f189d167b1dced14dc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.153828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d514aeb7ce148f189d167b1dced14dc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.174865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d514aeb7ce148f189d167b1dced14dc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.176749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d514aeb7ce148f189d167b1dced14dc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.195309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1f40b2a9-5459-4ca4-b5da-00da2fb5d8c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d514aeb7ce148f189d167b1dced14dc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.307691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1036/2070] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:56 2022] DELETE /identity/v3/users/0d514aeb7ce148f189d167b1dced14dc => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:57.364972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e301ec757c2344779a0d73f0136af8e1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.440024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e301ec757c2344779a0d73f0136af8e1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.477954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e301ec757c2344779a0d73f0136af8e1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.480322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e301ec757c2344779a0d73f0136af8e1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.623965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e301ec757c2344779a0d73f0136af8e1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:57.642056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1035/2071] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:57 2022] DELETE /identity/v3/users/e301ec757c2344779a0d73f0136af8e1 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:57.650090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:57.651055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:57.651529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:57.691070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:57.693607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:57.789231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:57.796489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1037/2072] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:37:57.832074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:57.833614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:57.984589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9085e9-2720-4ba1-96de-b97a1aae4c7e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:57.993079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:57.995755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:57.998597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:57.999702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:58.030359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:58.037985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3f947-c2a0-4f7e-85b9-580f94d6fc61 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:58.059923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1036/2073] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:37:58.532582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:58.535599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:58.709602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f445fd-e4c9-43bf-855b-9ea20ca81a27 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:58.709602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:58.709602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:58.709602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:58.710747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] PATH_INFO: `/v3/projects/09ca605da9fa46bda3362c53f34f1c76` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:58.710747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ca605da9fa46bda3362c53f34f1c76)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:58.710747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:58.957501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ca605da9fa46bda3362c53f34f1c76', 'request_id': 'req-d2effa4b-308e-4b1c-946a-66e0036dffbd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:58.966383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ca605da9fa46bda3362c53f34f1c76', 'request_id': 'req-d2effa4b-308e-4b1c-946a-66e0036dffbd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:58.973989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ca605da9fa46bda3362c53f34f1c76', 'request_id': 'req-d2effa4b-308e-4b1c-946a-66e0036dffbd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.066935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:59.079416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] The token cache is being invalidate because project 09ca605da9fa46bda3362c53f34f1c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:59.079416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.079416 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.081693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1038/2074] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:58 2022] DELETE /identity/v3/projects/09ca605da9fa46bda3362c53f34f1c76 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:37:59.083024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac5717-9998-450f-a4d2-4e1b2c93263f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.083319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac5717-9998-450f-a4d2-4e1b2c93263f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.083601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac5717-9998-450f-a4d2-4e1b2c93263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.121353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.121353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:59.150483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a2ac5717-9998-450f-a4d2-4e1b2c93263f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:59.152055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a2ac5717-9998-450f-a4d2-4e1b2c93263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:59.283613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effa4b-308e-4b1c-946a-66e0036dffbd admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:59.289608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:59.294457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.294457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.294457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] PATH_INFO: `/v3/projects/7055e2f1298f4b2386204d12c6d7871f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.294457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7055e2f1298f4b2386204d12c6d7871f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:59.296723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:59.296987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1037/2075] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:59.298697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.298983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.299258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.329843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:37:59.337752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:59.357414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1038/2076] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:37:59.363806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.383765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0cc7-63ce-4e94-ba5e-4de279472db6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:59.390888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:59.397127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.397804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.398364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.399429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:37:59.399834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:59.401689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:59.420871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1039/2077] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:37:59.425565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.437629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9604de1-09b9-4e9b-8a89-762a2cc4638b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:59.443307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:59.444291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.444754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.445184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.446237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:59.449032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:59.478729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:37:59.506789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7055e2f1298f4b2386204d12c6d7871f', 'request_id': 'req-366df003-7d46-42b8-9b7a-4b61a2a27390'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.516699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7055e2f1298f4b2386204d12c6d7871f', 'request_id': 'req-366df003-7d46-42b8-9b7a-4b61a2a27390'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.528182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7055e2f1298f4b2386204d12c6d7871f', 'request_id': 'req-366df003-7d46-42b8-9b7a-4b61a2a27390'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.620532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] The token cache is being invalidate because project 7055e2f1298f4b2386204d12c6d7871f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:37:59.621207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1040/2078] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:37:59 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:37:59.623616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.627589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.628447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:37:59.631566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1039/2079] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:37:59 2022] DELETE /identity/v3/projects/7055e2f1298f4b2386204d12c6d7871f => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:37:59.637271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.641122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:37:59.811225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366df003-7d46-42b8-9b7a-4b61a2a27390 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:59.832654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:59.832654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.832987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d274-f479-433d-8c50-8d75d89b3020 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:37:59.833135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.833135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] PATH_INFO: `/v3/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.833135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd3f1b3c9d354bcfa2a941e5f1ffbe5d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:59.835092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:59.836695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:37:59.837390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:37:59.837683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:37:59.837958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:37:59.839021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:37:59.842495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:37:59.965331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1041/2080] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:37:59 2022] POST /identity/v3/users => generated 285 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:37:59.968152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:37:59.971128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:00.016615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1952cc-97ac-4733-886c-d3b5523257a2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:00.025847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:00.027252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:00.027764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:00.028275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:00.029740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:00.032944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:00.055633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1042/2081] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:00.060426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:00.086551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6ef03-2ebf-4433-a54e-5d0a348ab6a4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:00.097560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:00.104893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:00.104893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:00.104893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:00.104893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:00.108508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:00.139299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1043/2082] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:00.149496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:00.153543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd3f1b3c9d354bcfa2a941e5f1ffbe5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:00.154487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd3f1b3c9d354bcfa2a941e5f1ffbe5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:00.176843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd3f1b3c9d354bcfa2a941e5f1ffbe5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:00.187053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd3f1b3c9d354bcfa2a941e5f1ffbe5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:00.187205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e6c5-67ed-4ee4-a365-ab0427d33b3d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:00.188729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-60d21fe4-79e0-4310-8516-62c6ae1a0074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd3f1b3c9d354bcfa2a941e5f1ffbe5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:00.188858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:00.190626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:00.191889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:00.193056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] PATH_INFO: `/v3/projects/69db07ee1ba648b48c08cf2409b1be6a/users/04d5b12e4e364934bd36f88fef8a2950/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:00.227298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69db07ee1ba648b48c08cf2409b1be6a, user_id=04d5b12e4e364934bd36f88fef8a2950, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:00.232054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:00.293219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1040/2083] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:37:59 2022] DELETE /identity/v3/users/cd3f1b3c9d354bcfa2a941e5f1ffbe5d => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:00.340764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1044/2084] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:00 2022] PUT /identity/v3/projects/69db07ee1ba648b48c08cf2409b1be6a/users/04d5b12e4e364934bd36f88fef8a2950/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:00.353807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:00.354361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:00.354921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:00.401453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] MFA Rules not processed for user `04d5b12e4e364934bd36f88fef8a2950`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:00.401453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:00.572770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1041/2085] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:00.588004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:00.594694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:00.786162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00901e03-a3c3-42bc-9e0f-762f5744817a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:00.799290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:00.800337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:00.800842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:00.801357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:00.802424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:00.814896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.045161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1045/2086] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:00 2022] POST /identity/v3/projects => generated 367 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:01.050451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.074087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dc761-ea72-4591-9917-8f1967ee092c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.167327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.168162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.168650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.169094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.170264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.172659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.344491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1042/2087] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:01 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:01.352554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.367136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de8f50-f158-4eab-b6bc-1cc2b2a2cc81 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.374440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.375496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.375985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.376473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.377455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.379818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.412755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1046/2088] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:01.418051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.434632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725204f-10c8-4a9b-8579-5a01c42370e0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.441237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.442231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.442519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.442821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] PATH_INFO: `/v3/projects/99ef029662f341bfb79ac63d4ab05386/users/f1215ede8edc47d5911ba120bf526426/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.476717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ef029662f341bfb79ac63d4ab05386, user_id=f1215ede8edc47d5911ba120bf526426, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.478177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.532672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.536227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:01.561646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265d1c-3c40-49e4-bb8e-97a82a746d1b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.581954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.581954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.581954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.581954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] PATH_INFO: `/v3/projects/309ed3708078464c8a97dabbe48b6cf9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.585258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=309ed3708078464c8a97dabbe48b6cf9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.585258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.616486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1043/2089] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:01 2022] PUT /identity/v3/projects/99ef029662f341bfb79ac63d4ab05386/users/f1215ede8edc47d5911ba120bf526426/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:01.620361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.670951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:01.687166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329965-9c33-48ee-a4ce-58340536e020 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.692087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.692730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.693010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.693282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.694026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.695666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.707056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1044/2090] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:01.711266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.722218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689669f7-9607-4698-94f2-1d7dfb526812 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.727089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.727979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.728263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.728647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] PATH_INFO: `/v3/projects/99ef029662f341bfb79ac63d4ab05386/users/f1215ede8edc47d5911ba120bf526426/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.730646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ef029662f341bfb79ac63d4ab05386, user_id=f1215ede8edc47d5911ba120bf526426, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.733370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9b669-4a8f-41e6-aaa6-0e9e6376d4c5 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.744591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1045/2091] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:01 2022] PUT /identity/v3/projects/99ef029662f341bfb79ac63d4ab05386/users/f1215ede8edc47d5911ba120bf526426/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:01.750744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309ed3708078464c8a97dabbe48b6cf9', 'request_id': 'req-641614d0-18fd-4d6b-b79d-99741a375873'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:01.756600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.756909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.757195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.758014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309ed3708078464c8a97dabbe48b6cf9', 'request_id': 'req-641614d0-18fd-4d6b-b79d-99741a375873'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:01.766065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309ed3708078464c8a97dabbe48b6cf9', 'request_id': 'req-641614d0-18fd-4d6b-b79d-99741a375873'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:01.798286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] MFA Rules not processed for user `f1215ede8edc47d5911ba120bf526426`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:01.803073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:01.864670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] The token cache is being invalidate because project 309ed3708078464c8a97dabbe48b6cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:01.865163 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:01.865821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-641614d0-18fd-4d6b-b79d-99741a375873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:01.867467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1047/2092] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:01 2022] DELETE /identity/v3/projects/309ed3708078464c8a97dabbe48b6cf9 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:01.872240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812dab0-3756-45c1-ab47-fdd500d1c62b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.874965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812dab0-3756-45c1-ab47-fdd500d1c62b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.874965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812dab0-3756-45c1-ab47-fdd500d1c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.875684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1046/2093] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:01.877145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:01.886658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:01.888084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:01.898949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8812dab0-3756-45c1-ab47-fdd500d1c62b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:01.900095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8812dab0-3756-45c1-ab47-fdd500d1c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:01.946916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d63a8c-e1e4-4418-ad8b-08da19040e92 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:01.951005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:01.952168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.952547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.952822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.954750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:01.962606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1048/2094] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:01.970917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:01.971113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:01.971326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:01.971606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:01.972669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:01.984767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1047/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:01.990373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:01.991805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:02.000570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1049/2096] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:02.005792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.017594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1370a6-3c40-44d2-86c0-0549c2ec439c service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.022593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.023245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.023547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.023821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.024574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:02.024878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.026536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.039896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1048/2097] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:02.044595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.059410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e727b-8400-4cc5-8ea9-c72156b5c148 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.064757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.065410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.065689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.065960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.066704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.068606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.153641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.155531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:02.167444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719a9b-493e-48c1-abe7-47c6c0cbf085 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.171202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.171837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.172115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.172390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.174125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:02.237372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.238830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.249315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1049/2098] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:02.275556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1050/2099] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:02 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:02.277304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.288912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d7969-a84e-453c-931b-e594220b6128 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.364515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.369389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.369389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.369389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.369389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.369389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.502918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1050/2100] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:02 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:02.507332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.528649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0aba1a-fd8a-49a5-a2dc-2a308e5cabc9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.536964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.545648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.545961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.546260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.547062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.548925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.566703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1051/2101] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:02.573132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.594662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21892-46e6-48c5-9c25-0770f695f932 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.603346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.608578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.608578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.608578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.608578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.610769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580c395-437c-418d-a738-6f80819efd3b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.625105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1051/2102] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:02.629779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.640215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa2f74-eec1-44fd-b7c0-f218b112d77b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.644814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.645695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.645967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.646267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] PATH_INFO: `/v3/projects/c9a3128f2c084fdcbd83984c59e9c7c2/users/d37ba9b15c7d4e8bae0befddde6fbb53/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.661936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a3128f2c084fdcbd83984c59e9c7c2, user_id=d37ba9b15c7d4e8bae0befddde6fbb53, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.664704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:02.786281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1052/2103] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:02 2022] PUT /identity/v3/projects/c9a3128f2c084fdcbd83984c59e9c7c2/users/d37ba9b15c7d4e8bae0befddde6fbb53/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:02.797414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.797683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.797956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.827044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] MFA Rules not processed for user `d37ba9b15c7d4e8bae0befddde6fbb53`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:02.828292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:02.900262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1052/2104] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:02.905653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:02.981490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4c3a1-7541-4332-9bb5-3efcd5756990 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:02.986624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:02.987289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:02.987569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:02.987841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:02.988536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:02.990605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.117149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1053/2105] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:02 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:03.119202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.134026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5535ad-307a-47f3-a897-c140c2302a15 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:03.227858 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:03.228642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.229032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.229430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.230677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:03.233960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.338083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1053/2106] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:03 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:03.342471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.354172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078af973-ca30-4879-92e6-01ddef9e086d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:03.359268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:03.359986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.360346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.360666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.361449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:03.363385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.376065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1054/2107] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:03.381460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.396097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358f705-6c99-4307-81d9-00ab58305158 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:03.403983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:03.405613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.406126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.406657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] PATH_INFO: `/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c/users/51e4eec2365942aabc7e45bfb4f582c6/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.433944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a1e4d1e42470ead1194a20c9dbe3c, user_id=51e4eec2365942aabc7e45bfb4f582c6, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:03.440101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.615618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1054/2108] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:03 2022] PUT /identity/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c/users/51e4eec2365942aabc7e45bfb4f582c6/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:03.618749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.635729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c221c-7684-447e-aa81-98368af42a0a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:03.716103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:03.716830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.717109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.717378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.718108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:03.719801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2b497-65c9-4d66-9761-c422d224c86f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.730127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1055/2109] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:03.735071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.746403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb590dd-4009-408a-a5ee-4ad3c483c609 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:03.752167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:03.753185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.753656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.754137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] PATH_INFO: `/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c/users/51e4eec2365942aabc7e45bfb4f582c6/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.756656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a1e4d1e42470ead1194a20c9dbe3c, user_id=51e4eec2365942aabc7e45bfb4f582c6, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:03.759860 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:03.771902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1055/2110] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:03 2022] PUT /identity/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c/users/51e4eec2365942aabc7e45bfb4f582c6/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:03.783541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:03.783808 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:03.784238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:03.818054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] MFA Rules not processed for user `51e4eec2365942aabc7e45bfb4f582c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:03.820363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:03.908570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:03.916665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1056/2111] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:03.928725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:03.996657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a982-2bf7-4c3f-b136-b75f96c62259 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:04.001053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:04.001768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:04.002047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:04.002364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:04.012107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:04.013591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:04.023835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1056/2112] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:04.137002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:04.139275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:04.154675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3c38-d824-4d5d-92cc-457bc14a36cf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:04.158688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:04.164081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:04.164081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:04.164081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:04.226694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:04.228186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9db90-194a-4015-8c98-6c1a12c4bad6 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:04.239557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1057/2113] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:04.511367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:06.610926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:06.954400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:07.317671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:07.974011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:08.645505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:08.786138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:08.787606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:08.853587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1220e-7dec-4f4e-87ad-b0d211c5ac0c service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:08.858061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:08.858972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:08.859255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:08.859528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] PATH_INFO: `/v3/users/04d5b12e4e364934bd36f88fef8a2950` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:08.860711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d5b12e4e364934bd36f88fef8a2950)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:08.862694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:09.088311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d5b12e4e364934bd36f88fef8a2950'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:09.089008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d5b12e4e364934bd36f88fef8a2950'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:09.101405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d5b12e4e364934bd36f88fef8a2950'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:09.102071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d5b12e4e364934bd36f88fef8a2950'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:09.110279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d5b12e4e364934bd36f88fef8a2950'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:09.142762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:09.186827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1057/2114] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:08 2022] DELETE /identity/v3/users/04d5b12e4e364934bd36f88fef8a2950 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:09.191448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:09.191735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:09.192012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:09.210654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:09.211964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:09.298420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1058/2115] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:09.308514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:09.310079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:09.391487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457826d5-43f7-43da-a4f7-5bc39d448bf0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:09.397035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:09.397727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:09.398015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:09.398349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:09.403312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:09.410555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:09.412275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:09.425084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1058/2116] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:09.582326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:09.935724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:09.937173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:09.954707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8d20a-3f4a-4838-a681-9276050465c5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:09.960311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:09.961325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:09.961585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:09.962886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] PATH_INFO: `/v3/projects/69db07ee1ba648b48c08cf2409b1be6a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:09.963197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69db07ee1ba648b48c08cf2409b1be6a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:09.965138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:09.993842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:10.098437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69db07ee1ba648b48c08cf2409b1be6a', 'request_id': 'req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.105216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69db07ee1ba648b48c08cf2409b1be6a', 'request_id': 'req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.111763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69db07ee1ba648b48c08cf2409b1be6a', 'request_id': 'req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.193609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] The token cache is being invalidate because project 69db07ee1ba648b48c08cf2409b1be6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:10.194321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.194576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.196100 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1059/2117] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:09 2022] DELETE /identity/v3/projects/69db07ee1ba648b48c08cf2409b1be6a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:10.200384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:10.206390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:10.211149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79effdf5-dd75-4987-9c27-9b312dcff78d service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:10.274385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:10.275300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:10.275570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:10.275839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] PATH_INFO: `/v3/users/f1215ede8edc47d5911ba120bf526426` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:10.277085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1215ede8edc47d5911ba120bf526426)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:10.279064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:10.552989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1215ede8edc47d5911ba120bf526426'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.553668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1215ede8edc47d5911ba120bf526426'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.567647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:10.572015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1215ede8edc47d5911ba120bf526426'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.572854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1215ede8edc47d5911ba120bf526426'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.580963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2b0b651b-711e-472e-b266-3bf63279f39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1215ede8edc47d5911ba120bf526426'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:10.691423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1059/2118] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:10 2022] DELETE /identity/v3/users/f1215ede8edc47d5911ba120bf526426 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:10.701727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:10.828237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:11.247270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:11.248833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:11.264371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770c6f-97c0-4348-bf95-7b3eb8f2f602 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:11.323783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:11.334690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:11.338170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:11.339117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:11.339428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:11.339809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] PATH_INFO: `/v3/projects/99ef029662f341bfb79ac63d4ab05386` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:11.341079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ef029662f341bfb79ac63d4ab05386)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:11.343820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:11.512857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ef029662f341bfb79ac63d4ab05386', 'request_id': 'req-27f89d99-2743-4cae-91fd-d1d5842bb662'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:11.520289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ef029662f341bfb79ac63d4ab05386', 'request_id': 'req-27f89d99-2743-4cae-91fd-d1d5842bb662'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:11.526501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ef029662f341bfb79ac63d4ab05386', 'request_id': 'req-27f89d99-2743-4cae-91fd-d1d5842bb662'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:11.662133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] The token cache is being invalidate because project 99ef029662f341bfb79ac63d4ab05386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:11.662686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:11.663823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-27f89d99-2743-4cae-91fd-d1d5842bb662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:11.666636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1060/2119] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:11 2022] DELETE /identity/v3/projects/99ef029662f341bfb79ac63d4ab05386 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:11.672241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:11.672809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:11.673394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:11.695255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:11.697109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:11.778804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1060/2120] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:11.788350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:11.788898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:11.789301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:11.816677 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:11.817976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:11.826911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1061/2121] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:11.831358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:11.844653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686a92-5834-49ef-96eb-6d8889b4973b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:11.853304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:11.854493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:11.854960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:11.855430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:11.857062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:11.857599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:11.860723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:11.886257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1061/2122] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:38:11.891555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:11.901518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef24c-666a-4936-893e-98f607a9bc96 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:11.906764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:11.908056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:11.908235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:11.908547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:11.909347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:11.911859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.077334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1062/2123] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:11 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:12.080197 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.101062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8c67b-5d0f-4c87-aa4c-ca6c076e20a4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:12.128771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:12.188503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:12.189308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.189525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.189894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.190716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:12.192987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.307075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1062/2124] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:12 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:12.312036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.322634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ac2ef-35f7-4fe3-ac34-c80ca0f69058 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:12.327185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:12.327850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.328132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.328408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.329136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:12.330901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.342800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1063/2125] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:12.347016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.359801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c951cc-2d30-493c-9fbf-f5f8bb411386 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:12.364577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:12.365252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.365534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.365813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.366670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:12.368296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ecddf-766f-4000-b6fb-08d7fce01af0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.379600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1063/2126] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:12.384324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.394870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca6b3f-1686-4c88-b1a4-6a0e697fb977 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:12.399541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:12.400446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.400724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.400996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] PATH_INFO: `/v3/projects/be61e4bb3db94e19b64534feb867b830/users/ea3a0d54e67f441e95cbb380dd891b8e/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.417838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be61e4bb3db94e19b64534feb867b830, user_id=ea3a0d54e67f441e95cbb380dd891b8e, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:12.421312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.520629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1064/2127] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:12 2022] PUT /identity/v3/projects/be61e4bb3db94e19b64534feb867b830/users/ea3a0d54e67f441e95cbb380dd891b8e/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:12.531974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.532247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.532565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.581251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] MFA Rules not processed for user `ea3a0d54e67f441e95cbb380dd891b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:12.582553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:12.679394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1064/2128] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:12.684327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.774351 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f9166-2e0f-47b8-a3cd-9bc840c3702e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:12.781379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:12.782228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:12.782526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:12.782799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:12.783539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:12.785596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:12.906850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:12.917876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1065/2129] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:12 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:12.919920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:12.944770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d99df-2566-4194-9661-4e7dc6b19190 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.021771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.022483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.022780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.023047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.023752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.025691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.135717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1065/2130] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:12 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:13.138145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:13.151085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dc096-cb3c-4fe8-b8d5-2896bc06e01c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.156427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.157138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.158141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.158535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.159784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.161650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.176523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1066/2131] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:13.183194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:13.195096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61be572-7506-41f2-aa0b-042785a1a43f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.202485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.202485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.202485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.202485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] PATH_INFO: `/v3/projects/679a32c5db654974aaa65f6747f1134a/users/5d6431ccbde541c48cbc3a9eb5275385/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.231664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] RBAC: Authorizing `identity:create_grant(project_id=679a32c5db654974aaa65f6747f1134a, user_id=5d6431ccbde541c48cbc3a9eb5275385, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.232626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.237259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:13.332342 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1066/2132] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:13 2022] PUT /identity/v3/projects/679a32c5db654974aaa65f6747f1134a/users/5d6431ccbde541c48cbc3a9eb5275385/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:13.336142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:13.349655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52ee51-37c2-497d-81d5-c567a0734f72 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.428346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.429417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.429753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.430228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.431007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.432895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d117dc5-e153-4d23-b721-857f9129b8b4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.445752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1067/2133] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:13.452453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:13.466006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036b2f0-31f4-4c95-ad09-70f9a3c0528b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.471599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.472522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.472805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.473082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] PATH_INFO: `/v3/projects/679a32c5db654974aaa65f6747f1134a/users/5d6431ccbde541c48cbc3a9eb5275385/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.475377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] RBAC: Authorizing `identity:create_grant(project_id=679a32c5db654974aaa65f6747f1134a, user_id=5d6431ccbde541c48cbc3a9eb5275385, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.478456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.491783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1067/2134] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:13 2022] PUT /identity/v3/projects/679a32c5db654974aaa65f6747f1134a/users/5d6431ccbde541c48cbc3a9eb5275385/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:13.503876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.504366 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.504838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.538187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] MFA Rules not processed for user `5d6431ccbde541c48cbc3a9eb5275385`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:13.539512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:13.624242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1068/2135] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:13.639208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:13.641557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:13.754863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac200b4a-7b41-4d56-99b0-25b790c6141e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:13.759907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:13.760780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:13.761125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:13.766312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:13.773543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:13.776502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:13.791598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1068/2136] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:14.168019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:14.168019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:14.172047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:14.182096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6e64-c313-4b0a-b1ff-970c56752289 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:14.186492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:14.187217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:14.187565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:14.187910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:14.247534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586d48e-377d-4986-bc31-f97785b10e07 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:14.252488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:14.253361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:14.253686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:14.253959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] PATH_INFO: `/v3/users/d37ba9b15c7d4e8bae0befddde6fbb53` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:14.255198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37ba9b15c7d4e8bae0befddde6fbb53)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:14.257066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:14.257259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:14.259990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3e935-4bc3-4f0b-8b8f-9524d877fef6 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:14.269881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:14.270959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1069/2137] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:14.594244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37ba9b15c7d4e8bae0befddde6fbb53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:14.604063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37ba9b15c7d4e8bae0befddde6fbb53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:14.619034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37ba9b15c7d4e8bae0befddde6fbb53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:14.619905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37ba9b15c7d4e8bae0befddde6fbb53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:14.776974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37ba9b15c7d4e8bae0befddde6fbb53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:14.787535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1069/2138] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:14 2022] DELETE /identity/v3/users/d37ba9b15c7d4e8bae0befddde6fbb53 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:14.792943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:14.792943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:14.792943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:14.827428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:14.829839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:14.854445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:14.856682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:14.923800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1070/2139] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:14.929744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:14.931834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:14.961872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:14.993593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4b02-e2cc-4eaf-bbb2-324ab7e7e61e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:14.999721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a8ff-0f8a-4103-ae61-86e430fdef12 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:15.004118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:15.004792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:15.005075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:15.005407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:15.016852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:15.018241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:15.018241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:15.018462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] PATH_INFO: `/v3/users/4f53a28588c24f1a8940c99be7e28917` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:15.020929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f53a28588c24f1a8940c99be7e28917)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:15.022407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:15.023861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:15.024281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:15.038797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1071/2140] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:15.351890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f53a28588c24f1a8940c99be7e28917'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.363105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f53a28588c24f1a8940c99be7e28917'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.400031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f53a28588c24f1a8940c99be7e28917'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.401665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f53a28588c24f1a8940c99be7e28917'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.533690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f53a28588c24f1a8940c99be7e28917'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.542991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1070/2141] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:14 2022] DELETE /identity/v3/users/4f53a28588c24f1a8940c99be7e28917 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:15.612913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:15.615509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:15.639786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa920f6d-9767-470a-a050-1b04a8a13711 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:15.743204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:15.744351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:15.744687 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:15.745049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] PATH_INFO: `/v3/projects/c9a3128f2c084fdcbd83984c59e9c7c2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:15.747530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a3128f2c084fdcbd83984c59e9c7c2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:15.750016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:15.893750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a3128f2c084fdcbd83984c59e9c7c2', 'request_id': 'req-fc77161d-6d6a-403b-b4d6-e515a8155510'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.901271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a3128f2c084fdcbd83984c59e9c7c2', 'request_id': 'req-fc77161d-6d6a-403b-b4d6-e515a8155510'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.908142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a3128f2c084fdcbd83984c59e9c7c2', 'request_id': 'req-fc77161d-6d6a-403b-b4d6-e515a8155510'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:15.918721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:15.921381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:15.939387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fe4dc-34e1-436a-b20f-458ed0a80ef1 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:15.944539 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:16.006493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:16.007419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:16.007732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:16.008008 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] PATH_INFO: `/v3/projects/6d2107609c264b7db70ffc4965ff45db` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:16.009634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2107609c264b7db70ffc4965ff45db)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:16.011672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:16.039922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] The token cache is being invalidate because project c9a3128f2c084fdcbd83984c59e9c7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:16.040426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.041337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.043542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1072/2142] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:15 2022] DELETE /identity/v3/projects/c9a3128f2c084fdcbd83984c59e9c7c2 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:16.048555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:16.050444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:16.074462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77161d-6d6a-403b-b4d6-e515a8155510 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:16.083666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:16.085351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:16.085861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:16.086406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] PATH_INFO: `/v3/users/51e4eec2365942aabc7e45bfb4f582c6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:16.088655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e4eec2365942aabc7e45bfb4f582c6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:16.092209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:16.159856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2107609c264b7db70ffc4965ff45db', 'request_id': 'req-02880b87-9f23-4ee8-97be-6585dc13d596'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.168725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2107609c264b7db70ffc4965ff45db', 'request_id': 'req-02880b87-9f23-4ee8-97be-6585dc13d596'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.178057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2107609c264b7db70ffc4965ff45db', 'request_id': 'req-02880b87-9f23-4ee8-97be-6585dc13d596'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.258702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] The token cache is being invalidate because project 6d2107609c264b7db70ffc4965ff45db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:16.268023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.268023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.268023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1071/2143] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:15 2022] DELETE /identity/v3/projects/6d2107609c264b7db70ffc4965ff45db => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:16.268023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:16.268023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:16.336905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880b87-9f23-4ee8-97be-6585dc13d596 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:16.342007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:16.342945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:16.343239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:16.343515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] PATH_INFO: `/v3/users/2f0d58b741454850acf20bedb706a5a8` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:16.344889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0d58b741454850acf20bedb706a5a8)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:16.346975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:16.412386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e4eec2365942aabc7e45bfb4f582c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.413384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e4eec2365942aabc7e45bfb4f582c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.427353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:16.430369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e4eec2365942aabc7e45bfb4f582c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.431269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e4eec2365942aabc7e45bfb4f582c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.444734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e4eec2365942aabc7e45bfb4f582c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.583132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1073/2144] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:16 2022] DELETE /identity/v3/users/51e4eec2365942aabc7e45bfb4f582c6 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:16.691806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0d58b741454850acf20bedb706a5a8'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.702640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0d58b741454850acf20bedb706a5a8'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.718637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0d58b741454850acf20bedb706a5a8'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.719384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0d58b741454850acf20bedb706a5a8'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.802449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0d58b741454850acf20bedb706a5a8'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:16.811835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1072/2145] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:16 2022] DELETE /identity/v3/users/2f0d58b741454850acf20bedb706a5a8 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:17.220765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:17.223014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:17.233131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:17.235096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:17.322802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f80ed-7f9b-467d-b5ba-83bd5bbf174d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:17.359876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb92a-a352-422b-a7ae-7f07c43e34a5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:17.365794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:17.367245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:17.373530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:17.373530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] PATH_INFO: `/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:17.373530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=796a1e4d1e42470ead1194a20c9dbe3c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:17.373530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:17.444651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:17.447839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:17.447839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:17.448994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] PATH_INFO: `/v3/projects/7216644e33b64343ab6f3f631731fa08` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:17.452042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7216644e33b64343ab6f3f631731fa08)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:17.456198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:17.661836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796a1e4d1e42470ead1194a20c9dbe3c', 'request_id': 'req-00b996b4-bf0d-4a7d-894c-2944f719c2cf'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.669237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796a1e4d1e42470ead1194a20c9dbe3c', 'request_id': 'req-00b996b4-bf0d-4a7d-894c-2944f719c2cf'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.675941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796a1e4d1e42470ead1194a20c9dbe3c', 'request_id': 'req-00b996b4-bf0d-4a7d-894c-2944f719c2cf'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.713819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7216644e33b64343ab6f3f631731fa08', 'request_id': 'req-1cb17396-a428-4df1-81e1-83399d723962'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.721039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7216644e33b64343ab6f3f631731fa08', 'request_id': 'req-1cb17396-a428-4df1-81e1-83399d723962'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.727307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7216644e33b64343ab6f3f631731fa08', 'request_id': 'req-1cb17396-a428-4df1-81e1-83399d723962'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.781967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] The token cache is being invalidate because project 796a1e4d1e42470ead1194a20c9dbe3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:17.782510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.783588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.786382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1073/2146] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:17 2022] DELETE /identity/v3/projects/796a1e4d1e42470ead1194a20c9dbe3c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:17.787878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:17.790184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:17.879462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b996b4-bf0d-4a7d-894c-2944f719c2cf admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:17.913986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] The token cache is being invalidate because project 7216644e33b64343ab6f3f631731fa08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:17.914387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.915308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1cb17396-a428-4df1-81e1-83399d723962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:17.923618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1074/2147] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:17 2022] DELETE /identity/v3/projects/7216644e33b64343ab6f3f631731fa08 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:17.926227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05a0d-bf62-4edc-aea7-e196d5f05cc8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:17.926752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05a0d-bf62-4edc-aea7-e196d5f05cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:17.930794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff05a0d-bf62-4edc-aea7-e196d5f05cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:17.960975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:17.962589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:17.963097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:17.971038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] PATH_INFO: `/v3/users/5ec4a03199344617bb80c589de6d0273` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:17.971038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec4a03199344617bb80c589de6d0273)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:17.971413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:17.973842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cff05a0d-bf62-4edc-aea7-e196d5f05cc8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:17.975413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cff05a0d-bf62-4edc-aea7-e196d5f05cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:17.985272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1075/2148] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:17.987492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b761a9-e00b-4260-8c53-55dccdfa030b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:17.987855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b761a9-e00b-4260-8c53-55dccdfa030b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:17.988204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b761a9-e00b-4260-8c53-55dccdfa030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.018189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f5b761a9-e00b-4260-8c53-55dccdfa030b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:18.019554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f5b761a9-e00b-4260-8c53-55dccdfa030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.027862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1076/2149] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:18.029239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6d58-ee78-407e-a12e-d64d1f014800 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.033994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6d58-ee78-407e-a12e-d64d1f014800 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.033994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d6d58-ee78-407e-a12e-d64d1f014800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.049950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-716d6d58-ee78-407e-a12e-d64d1f014800 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:18.051335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-716d6d58-ee78-407e-a12e-d64d1f014800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.061676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1077/2150] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:18.063517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.063859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.073480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.093671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:18.095958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.098656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:18.106091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1078/2151] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:18.107535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.117390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f6ed9-5989-4bc8-95a1-ffb074649ecf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.122354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.123600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.124021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.124812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.124949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:18.125163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.126845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.139797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1079/2152] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:18.140673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.143797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.156753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf5735-11f4-436a-b60e-2a4cc9532529 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.161748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.162663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.162940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.163212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] PATH_INFO: `/v3/users/8c12eb4d4c924e2ab0350249fe35d0da` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.169984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c12eb4d4c924e2ab0350249fe35d0da)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.169984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.228414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec4a03199344617bb80c589de6d0273'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.243076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec4a03199344617bb80c589de6d0273'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.266151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec4a03199344617bb80c589de6d0273'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.267234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec4a03199344617bb80c589de6d0273'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.381958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec4a03199344617bb80c589de6d0273'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.388752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1074/2153] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:17 2022] DELETE /identity/v3/users/5ec4a03199344617bb80c589de6d0273 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:18.389684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.391106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.438455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c12eb4d4c924e2ab0350249fe35d0da'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.439594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c12eb4d4c924e2ab0350249fe35d0da'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.453173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c12eb4d4c924e2ab0350249fe35d0da'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.454262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c12eb4d4c924e2ab0350249fe35d0da'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.462558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1ac4900a-8533-4f2b-b4bc-c7cc040ae50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c12eb4d4c924e2ab0350249fe35d0da'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:18.466096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438d8b3-cccb-4490-bee8-232ea893d914 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.474072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.475285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.475773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.476261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.477662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:18.478127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.481352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.503891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1075/2154] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:18.507832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.510930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.532927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869cb9c-b859-45d2-816a-a87cd3a33bf6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.537898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.540227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.540227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.540227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.540227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.541721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151c43-812d-4a29-94f1-6eb262a7c969 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.574630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1080/2155] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:18 2022] DELETE /identity/v3/users/8c12eb4d4c924e2ab0350249fe35d0da => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:18.577909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.577909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.577909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.596019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:18.597337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.670074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1081/2156] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:18.671503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.673844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.680285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1076/2157] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:18.683115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.683663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.684344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.687338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcdfd0-e2c1-4972-8c55-f09d71cfa7c0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.708817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:18.717726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:18.723185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.770415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.771282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.771691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.771976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.772769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.775009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.799407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1077/2158] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:18.799909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.801388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:18.873818 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7aad-f0ac-4c8b-945b-7c5efc154127 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:18.878429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:18.879098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:18.879392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:18.879650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:18.896375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:18.897932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:18.907383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1082/2159] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:18.908868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.914710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1078/2160] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:18.922740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:18.933142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc286e0-1028-46d0-af6b-0396eb00cd11 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.003034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.004660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.005295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.005889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.010695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0d72d-de50-45c7-be81-ed9679d08e4d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.016795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.017471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.018412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.018412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.018740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.020931 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.022295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.024904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.038856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1079/2161] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:19.041656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.050586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076c041-a9c8-43e6-ba6e-5f9e915cd507 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.055756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.056424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.056725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.057063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.057815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.059832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.132928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1083/2162] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:18 2022] POST /identity/v3/users => generated 291 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:19.137707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.149485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964941-34e8-4802-88e7-876ee91d8107 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.154596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.155265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.155552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.155826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.156555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.158184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.169617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1084/2163] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:19.174099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.178518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1080/2164] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:19 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:19.182699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.187404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41db1cb-4008-4eac-a2d6-9cdbadee0ea3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.196072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1ed8a-0ee0-415a-b98a-53af90ca7e75 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.198871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.199846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.200327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.201242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.202064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.204409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.204507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.204955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.205243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.205549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.206324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.208397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.221306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1081/2165] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:19.225267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1085/2166] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:19.226692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.227080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.239108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb95136-f238-4d97-9e7e-03a7bf709d3b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.245373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.248202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cc765-5f48-40a7-b946-bdd9aa9d2a79 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.248373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.248373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.248373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.248373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.250149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.256348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.257907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.258530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.259150 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] PATH_INFO: `/v3/projects/c410b143396148f9b1a047d991f164dc/users/0e7015de90a84430adf95c7e0012ab49/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.265196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1082/2167] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:19.270790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.283197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9788-02be-41a6-9223-0da1d4e7dc28 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.283839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c410b143396148f9b1a047d991f164dc, user_id=0e7015de90a84430adf95c7e0012ab49, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.288388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95762d-785b-4480-9a48-577bd03d4084 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.288663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.289601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.289944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.290269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] PATH_INFO: `/v3/projects/223131551a1149bd86a19865247c9953/users/5e16e36e7b8943b2af671037bc63d2e4/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.313665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] RBAC: Authorizing `identity:create_grant(project_id=223131551a1149bd86a19865247c9953, user_id=5e16e36e7b8943b2af671037bc63d2e4, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.318491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b465aa-427b-4d73-b457-bc5ca2b9d49e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.483697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1086/2168] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:19 2022] PUT /identity/v3/projects/c410b143396148f9b1a047d991f164dc/users/0e7015de90a84430adf95c7e0012ab49/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:19.500436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.501306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.501306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.518383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1083/2169] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:19 2022] PUT /identity/v3/projects/223131551a1149bd86a19865247c9953/users/5e16e36e7b8943b2af671037bc63d2e4/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:19.535227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.535494 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.535779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.542292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] MFA Rules not processed for user `0e7015de90a84430adf95c7e0012ab49`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:19.543501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:19.565605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] MFA Rules not processed for user `5e16e36e7b8943b2af671037bc63d2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:19.567006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:19.625701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1087/2170] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:19.627261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.631443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:19.686170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1084/2171] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:19.688996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:19.698253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:19.733385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817c9e8-0d94-485d-ba8e-eabdd187403a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.740964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.742095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.742567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.742899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] PATH_INFO: `/v3/projects/dcf66f7181f34c6baa3fe60a124d6a12` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.744218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf66f7181f34c6baa3fe60a124d6a12)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.746213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.784135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da8bcf-8004-45f1-a4c9-8faf53231fd9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:19.789948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:19.790913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:19.791195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:19.791470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] PATH_INFO: `/v3/projects/7cfedfba7b154c0999c294601945d813` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:19.792682 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cfedfba7b154c0999c294601945d813)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:19.795586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:19.917770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf66f7181f34c6baa3fe60a124d6a12', 'request_id': 'req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:19.925256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf66f7181f34c6baa3fe60a124d6a12', 'request_id': 'req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:19.931798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf66f7181f34c6baa3fe60a124d6a12', 'request_id': 'req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:19.957301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cfedfba7b154c0999c294601945d813', 'request_id': 'req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:19.967148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cfedfba7b154c0999c294601945d813', 'request_id': 'req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:19.973380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cfedfba7b154c0999c294601945d813', 'request_id': 'req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.065119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] The token cache is being invalidate because project dcf66f7181f34c6baa3fe60a124d6a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:20.065350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.066031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.067746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1088/2172] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:19 2022] DELETE /identity/v3/projects/dcf66f7181f34c6baa3fe60a124d6a12 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:20.069363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:20.070930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.123082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] The token cache is being invalidate because project 7cfedfba7b154c0999c294601945d813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:20.123528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.124375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.126478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1085/2173] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:19 2022] DELETE /identity/v3/projects/7cfedfba7b154c0999c294601945d813 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:20.127436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:20.129452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.133477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3209-0bd9-43bd-b1b3-27ecd0f37a66 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:20.199151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:20.199808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.200099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.200353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.203008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.207223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:20.214704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ce24-f903-4ed3-be4f-a1fc872d57f6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:20.221665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:20.222848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.223361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.223863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.227679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.278630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:20.280979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:20.290902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1089/2174] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:20.291841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:20.293220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.295732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:20.297207 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46680f78-a3d5-4832-98c9-f4f3a59ca979 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:20.310681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1086/2175] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:20.311174 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.314601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.314601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.343945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:20.345165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.381768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332974b-c49b-4333-9341-8a76b5d3145d service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:20.387033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:20.387925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.388780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.389086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] PATH_INFO: `/v3/users/adcefcef3036414a8bcc93bca5c7c84a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.390398 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] RBAC: Authorizing `identity:delete_user(user_id=adcefcef3036414a8bcc93bca5c7c84a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:20.392593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:20.413907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1087/2176] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:20.414855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:20.416162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.427851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d042d82-b706-4c48-b566-f324f597ff33 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:20.433220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:20.434176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.434480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.434856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] PATH_INFO: `/v3/users/0e7015de90a84430adf95c7e0012ab49` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.440795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7015de90a84430adf95c7e0012ab49)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:20.440795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:20.504756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:20.703259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adcefcef3036414a8bcc93bca5c7c84a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.704045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adcefcef3036414a8bcc93bca5c7c84a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.720724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adcefcef3036414a8bcc93bca5c7c84a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.722842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adcefcef3036414a8bcc93bca5c7c84a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.737265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7015de90a84430adf95c7e0012ab49'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.738663 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adcefcef3036414a8bcc93bca5c7c84a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.749282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7015de90a84430adf95c7e0012ab49'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.768092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7015de90a84430adf95c7e0012ab49'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.768889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7015de90a84430adf95c7e0012ab49'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.875682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1090/2177] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:20 2022] DELETE /identity/v3/users/adcefcef3036414a8bcc93bca5c7c84a => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:20.880119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:20.880119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:20.939358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7bb81657-5346-45b0-b4e3-7a9824612a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7015de90a84430adf95c7e0012ab49'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:20.951636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1088/2178] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:20 2022] DELETE /identity/v3/users/0e7015de90a84430adf95c7e0012ab49 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:20.951636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1119-1c0a-42ed-8b17-c78a6fca9daa None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:20.951636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1119-1c0a-42ed-8b17-c78a6fca9daa None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:20.952011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1119-1c0a-42ed-8b17-c78a6fca9daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:20.977025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ea0c1119-1c0a-42ed-8b17-c78a6fca9daa None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:20.978711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ea0c1119-1c0a-42ed-8b17-c78a6fca9daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.000245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab21702-e127-459d-8a7e-05af8b78801b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.063702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1089/2179] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:21.065290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.065573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.065851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.086230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:21.087226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.087448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.088364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.089264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.089264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] PATH_INFO: `/v3/users/5e16e36e7b8943b2af671037bc63d2e4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.090484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e16e36e7b8943b2af671037bc63d2e4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.092579 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.096388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1090/2180] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:21.097236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.108660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be2fe4-ab34-4b9b-ba29-ff2c57130478 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.114250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.114931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.115208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.115480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.116214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:21.116579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.118457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.133127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1091/2181] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:21.134027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.140591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.243211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593714d-7e8a-4ae2-b65f-0c6221211040 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.247981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.248813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.249169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.249511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.261309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.262904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.276272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1092/2182] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:21.276551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.288537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b36d8-dd39-46aa-adab-a91442df2315 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.295413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.296238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.296832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.296976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.297772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.300120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.320806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e16e36e7b8943b2af671037bc63d2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.321698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e16e36e7b8943b2af671037bc63d2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.349112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e16e36e7b8943b2af671037bc63d2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.349997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e16e36e7b8943b2af671037bc63d2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.361322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ae0b68b8-d5bb-4fbb-8170-46b066f9403e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e16e36e7b8943b2af671037bc63d2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.418382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1093/2183] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:21 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:21.420052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.421931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.489832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e947f3-17b1-4cac-a09d-b48d081f73e5 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.494870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.495548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.495833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.496108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.496804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.498920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.536558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1091/2184] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:20 2022] DELETE /identity/v3/users/5e16e36e7b8943b2af671037bc63d2e4 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:21.542723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0c553-2704-4317-89a6-669f14120365 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.543105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0c553-2704-4317-89a6-669f14120365 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.543386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0c553-2704-4317-89a6-669f14120365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.562932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ced0c553-2704-4317-89a6-669f14120365 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:21.564169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ced0c553-2704-4317-89a6-669f14120365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.635888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1092/2185] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:21.637143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0c553-2704-4317-89a6-669f14120365 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.658861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1094/2186] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:21 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:21.659971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.661115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ced0c553-2704-4317-89a6-669f14120365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.663784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:21.671596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:21.682783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0c553-2704-4317-89a6-669f14120365 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.695439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.696808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.698679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.698679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] PATH_INFO: `/v3/projects/506a399779b2447fa1d3a2b70ad73880` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.700190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=506a399779b2447fa1d3a2b70ad73880)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.705424 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.758387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41434-3c97-4d0d-a05e-ce00025e1ca8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.777769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:21.824307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.826293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.826810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.827313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.842978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.844696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.855714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1095/2187] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:21.856579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.866841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da7e9e-3b1b-43d9-83cc-8ba2e3e546fa service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.871785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.874922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.874922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.874922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.874922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.875571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.887806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1096/2188] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:21.892966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.905158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069211-7e29-4125-810d-8690ab52a6a2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.912715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:21.913555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:21.915187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:21.915187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:21.915187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:21.916609 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:21.929294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1097/2189] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:21.936634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:21.952615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dc296-f82f-4a64-8751-dad7ada5b728 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:21.952983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506a399779b2447fa1d3a2b70ad73880', 'request_id': 'req-449100a0-2f53-4200-9611-b776030a0ace'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.965480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506a399779b2447fa1d3a2b70ad73880', 'request_id': 'req-449100a0-2f53-4200-9611-b776030a0ace'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:21.977595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506a399779b2447fa1d3a2b70ad73880', 'request_id': 'req-449100a0-2f53-4200-9611-b776030a0ace'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.038094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.039248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.039685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.040141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] PATH_INFO: `/v3/projects/0e6367d6524b4a999e92138487442177/users/00d7efeeb0cb44c5a4c7027b69d7286d/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.058009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6367d6524b4a999e92138487442177, user_id=00d7efeeb0cb44c5a4c7027b69d7286d, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.061516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa30d-b5a3-4014-86ba-13138111e34e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.149991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] The token cache is being invalidate because project 506a399779b2447fa1d3a2b70ad73880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:22.150336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.151044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.153302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1093/2190] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:21 2022] DELETE /identity/v3/projects/506a399779b2447fa1d3a2b70ad73880 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:22.153932 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:22.155367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.174055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449100a0-2f53-4200-9611-b776030a0ace admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:22.182256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.183518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.184058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.184568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] PATH_INFO: `/v3/projects/c410b143396148f9b1a047d991f164dc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.186630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c410b143396148f9b1a047d991f164dc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.189481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.227676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1098/2191] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:21 2022] PUT /identity/v3/projects/0e6367d6524b4a999e92138487442177/users/00d7efeeb0cb44c5a4c7027b69d7286d/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:22.230454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4316-15fd-45a4-a94c-e7c14d9cc67c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.230963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4316-15fd-45a4-a94c-e7c14d9cc67c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.231456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4316-15fd-45a4-a94c-e7c14d9cc67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.262535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-16ca4316-15fd-45a4-a94c-e7c14d9cc67c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:22.264563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-16ca4316-15fd-45a4-a94c-e7c14d9cc67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.360181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1099/2192] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:22.361365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f8af-8b74-420f-96e2-db8cca2b9d41 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.361658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f8af-8b74-420f-96e2-db8cca2b9d41 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.361937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f8af-8b74-420f-96e2-db8cca2b9d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.387260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c410b143396148f9b1a047d991f164dc', 'request_id': 'req-efc49b95-2736-4de3-8096-e9934f85edd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.400046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c410b143396148f9b1a047d991f164dc', 'request_id': 'req-efc49b95-2736-4de3-8096-e9934f85edd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.402404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5f18f8af-8b74-420f-96e2-db8cca2b9d41 None None] MFA Rules not processed for user `00d7efeeb0cb44c5a4c7027b69d7286d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:22.403838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5f18f8af-8b74-420f-96e2-db8cca2b9d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.420294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c410b143396148f9b1a047d991f164dc', 'request_id': 'req-efc49b95-2736-4de3-8096-e9934f85edd8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.516118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1100/2193] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:22.518111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6ed0d-0600-47d0-a29b-b6a6fba0861f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.518787 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6ed0d-0600-47d0-a29b-b6a6fba0861f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.519220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6ed0d-0600-47d0-a29b-b6a6fba0861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.543565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-94e6ed0d-0600-47d0-a29b-b6a6fba0861f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:22.543903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] The token cache is being invalidate because project c410b143396148f9b1a047d991f164dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:22.544882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.545595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.547078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-94e6ed0d-0600-47d0-a29b-b6a6fba0861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.550758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1094/2194] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:22 2022] DELETE /identity/v3/projects/c410b143396148f9b1a047d991f164dc => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:22.550758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:22.550758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.635709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1101/2195] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:22.638071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.639039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.639545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.654051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49b95-2736-4de3-8096-e9934f85edd8 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:22.659125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.659922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.660309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.661049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.663497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.672377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:22.675612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.679750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.681359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.688455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1102/2196] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:22.691354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:22.693527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.696138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1095/2197] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:22.697013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:22.709239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc188-fd24-4517-9079-f5e4eff9cf5e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:22.717225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.718690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.720541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.720541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] PATH_INFO: `/v3/projects/223131551a1149bd86a19865247c9953` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.720858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223131551a1149bd86a19865247c9953)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.722866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.725155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c49c3-aea0-468d-909d-643c7441cd7c service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:22.733899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.734863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.735221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.735511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.736228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:22.736558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.738513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a3f33-e124-48fd-8f8a-51ce9a2ec366 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.763487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1096/2198] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:22.767532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.767911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.768260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.788136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:22.789474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:22.802138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1097/2199] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:22.802138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:22.812177 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938373-9169-465e-9b1c-8bb777a49e7e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:22.819829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:22.820677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:22.821179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:22.821179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:22.822235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:22.824195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:22.853821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:22.882227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223131551a1149bd86a19865247c9953', 'request_id': 'req-81c9c977-0199-4e7c-af11-13bf35fcda15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.895838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223131551a1149bd86a19865247c9953', 'request_id': 'req-81c9c977-0199-4e7c-af11-13bf35fcda15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.900819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223131551a1149bd86a19865247c9953', 'request_id': 'req-81c9c977-0199-4e7c-af11-13bf35fcda15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:22.961662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1098/2200] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:22 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:22.962609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.024429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672f367-69f9-47f6-a437-ba39b5adfe73 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.031917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.033130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.033625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.034080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.035400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:23.035901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.039141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6ac6-ebd1-44ad-a158-26eafcdaf9e3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.043811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] The token cache is being invalidate because project 223131551a1149bd86a19865247c9953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:23.044106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:23.045402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:23.047038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1103/2201] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:22 2022] DELETE /identity/v3/projects/223131551a1149bd86a19865247c9953 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:23.050644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.055825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:23.057704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1099/2202] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:23.058689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.065089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.065089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.090800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:23.093372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:23.104676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1100/2203] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:23.105363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.108175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:23.122593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90897-245a-4142-807d-9e4aef2c81f1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.128354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.129071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.129358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.129656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.130359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.132404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.151493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9c977-0199-4e7c-af11-13bf35fcda15 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.156914 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.157601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.157878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.158151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.158952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.160954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51be86-2b60-4966-8de3-53e0b99ace31 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.301346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1104/2204] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:23.306369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1101/2205] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:23.306546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.306546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.306546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.306949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.339758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:23.352726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:23.354537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:23.427067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2a66d-4eaa-4da5-9aa8-a096e6bba91a admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.436308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.437999 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.438423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.438743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.439598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.441365 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.458820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1102/2206] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:23.461791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.461923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1105/2207] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:23.463034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.476026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0541a-26b8-4acd-afe8-2c2d7af07ba7 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.478482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8cc2-d9d1-47ef-b8d3-241ec289a190 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.483163 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.484189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.484649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.485123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.485657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.486300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.486693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.487141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.487543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.488744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.489682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.492082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.508338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1106/2208] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:23.509347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.523352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044aa5e-4b7e-49aa-9819-c5d3ba876350 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.531481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.532685 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.533223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.533897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.535244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:23.535800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.539189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.556531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1107/2209] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:23.557372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.574730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b0307-7ae0-459c-9406-b958ed44eb9a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.583978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.586053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.586614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.587081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] PATH_INFO: `/v3/projects/afe9e3392df044e5aa0c8627c179e807/users/4245fc4b5ad948898537af66820cca67/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.604322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe9e3392df044e5aa0c8627c179e807, user_id=4245fc4b5ad948898537af66820cca67, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.607214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.726069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1103/2210] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/users => generated 281 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:23.737796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.740513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1108/2211] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:23 2022] PUT /identity/v3/projects/afe9e3392df044e5aa0c8627c179e807/users/4245fc4b5ad948898537af66820cca67/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:23.745710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.758962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd47ce9-16e1-48f7-aaa0-55bea4ccb03e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.841210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbab45-7dfa-431f-9561-ef4af2707ab9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:23.846131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.846937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.847286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.847567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.848314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.849978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae8af-5c80-4df5-af7a-e3cd7ec2afd2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.855426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:23.856095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.856396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.856781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.857490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:23.859614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:23.864180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1109/2212] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:23.865562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:23.865855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:23.866521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:23.919335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] MFA Rules not processed for user `4245fc4b5ad948898537af66820cca67`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:23.920714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:23.984885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1104/2213] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:23.986015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:23.997037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1110/2214] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:23.998857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.000335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.014958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc5e27-3bf5-48b8-b48f-d16b780ed1a5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.053025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73f8a5-d77e-4fd8-a079-3e4a19592258 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.058421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.059129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.059414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.059694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.061068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.063396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.073447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1105/2215] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:24.075028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.077771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.078793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.079170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.079547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] PATH_INFO: `/v3/users/ea3a0d54e67f441e95cbb380dd891b8e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.080861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3a0d54e67f441e95cbb380dd891b8e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.083072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.086358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626b1a2-6a2a-4bef-9e60-148f57ed78d4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.091427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.092080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.092361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.092666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.093362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.095565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.195475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1106/2216] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:24 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:24.195475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.203366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba79c-dfff-4413-b3c5-effa12f9393c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.209477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.210151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.213744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.213744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.213744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.213744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.300499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3a0d54e67f441e95cbb380dd891b8e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:24.308099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3a0d54e67f441e95cbb380dd891b8e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:24.321701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3a0d54e67f441e95cbb380dd891b8e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:24.322489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3a0d54e67f441e95cbb380dd891b8e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:24.326658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1107/2217] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:24 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:24.327711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.402078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4962b-5a55-41ce-9c93-c8ee77c1e978 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.407809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.408820 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.409144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.409463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] PATH_INFO: `/v3/projects/be45be35531949e4bb49f81dd456223b/users/9a25e3a0b0944de7ad06005733861759/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.427106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be45be35531949e4bb49f81dd456223b, user_id=9a25e3a0b0944de7ad06005733861759, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.430123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.456414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3a0d54e67f441e95cbb380dd891b8e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:24.466992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1111/2218] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:23 2022] DELETE /identity/v3/users/ea3a0d54e67f441e95cbb380dd891b8e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:24.468670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.471522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.494832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:24.568423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93cecf-6e66-4b70-877f-70faf99b2d32 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.577080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.578262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.579509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.579509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.580664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.583833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38671cb7-21da-47ab-acac-f43159a74b14 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.593444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1108/2219] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:24 2022] PUT /identity/v3/projects/be45be35531949e4bb49f81dd456223b/users/9a25e3a0b0944de7ad06005733861759/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:24.604151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.610373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1112/2220] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:24.610373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.610373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.610373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.614048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.628852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:24.630089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.686327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:24.702285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1113/2221] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:24.703270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.708813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5306-7987-4d45-92db-9b8a38aff027 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.713453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ceb393-2240-4dc3-8505-085cb929a4f1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.717173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.718912 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.726484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.726484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.727186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.727312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.727401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.727401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.727401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.727401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.727714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.727797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc506cc1-9ae4-4137-88c3-eb288ab4433e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.742617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1114/2222] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:24.743939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.744221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.744504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.774608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] MFA Rules not processed for user `9a25e3a0b0944de7ad06005733861759`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:24.775964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.848415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1109/2223] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:24 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:24.848415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.848852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1115/2224] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:24.848852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.862479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:24.893177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce8676-585e-40ab-a862-7c3ba6a9ae65 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.908774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.910866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.918368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.919341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] PATH_INFO: `/v3/projects/dd8de9d9daad459d90f784a79173b759/users/9971ea7b401542e3b1a7efe3a897f64e/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.943118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8de9d9daad459d90f784a79173b759, user_id=9971ea7b401542e3b1a7efe3a897f64e, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.943525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118b57-91f0-4595-b4ac-844cc5beb86d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.946638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.947408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.948065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:24.948478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:24.948631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:24.959537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:24.960989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:24.975051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1110/2225] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:24.976007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:24.985725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8837e-4b5b-495e-99ed-e2fff0b50582 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:24.999013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:24.999775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.000262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.000739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.001701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.004356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.019249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1111/2226] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:25.024046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.025929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:25.065502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1116/2227] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:24 2022] PUT /identity/v3/projects/dd8de9d9daad459d90f784a79173b759/users/9971ea7b401542e3b1a7efe3a897f64e/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:25.067447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.133064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83e9fe-ee52-427c-b6cb-06288df002fc admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.172605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d656-91ed-4b37-9644-731957382625 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.177739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.178986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.179227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.179504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] PATH_INFO: `/v3/projects/2b44322704dd4d8899f770544b420b7f/users/76d218362bc246f1bae64a0611d3e5a2/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.192690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.193381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.193667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.193992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.197830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b44322704dd4d8899f770544b420b7f, user_id=76d218362bc246f1bae64a0611d3e5a2, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.200176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.265467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.267051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff34cc-0158-4c9b-bc48-a558b1d1eae7 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.278973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1112/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:25.280715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.281015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.281300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.311597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1117/2229] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:25 2022] PUT /identity/v3/projects/2b44322704dd4d8899f770544b420b7f/users/76d218362bc246f1bae64a0611d3e5a2/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:25.312644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.317688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:25.336364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] MFA Rules not processed for user `9971ea7b401542e3b1a7efe3a897f64e`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:25.338039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:25.484636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1113/2230] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:25.486625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.489047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc188c2-ba93-41ef-8d0f-eb747cedba3a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.495582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.496866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.497387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.503433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a0a9d-f651-423f-a3c4-c0f102e84e26 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.505516 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] PATH_INFO: `/v3/users/9a25e3a0b0944de7ad06005733861759` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.511359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a25e3a0b0944de7ad06005733861759)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.511359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.511552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.511552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.511552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.511552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.512186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.514020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.531531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1114/2231] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:25.533344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.535943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:25.625709 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ff450-3a22-4db6-bb95-04ec550c50ad admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.632390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.633509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.633901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.634373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.650593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.653191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.675968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1115/2232] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:25.677431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.696781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c44d-e22b-4d2d-a932-68620bffb90d service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.721013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.724141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.730743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.731378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] PATH_INFO: `/v3/projects/2b44322704dd4d8899f770544b420b7f/users/76d218362bc246f1bae64a0611d3e5a2/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.738541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b44322704dd4d8899f770544b420b7f, user_id=76d218362bc246f1bae64a0611d3e5a2, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.749658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.762725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a25e3a0b0944de7ad06005733861759'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:25.773780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1116/2233] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:25 2022] PUT /identity/v3/projects/2b44322704dd4d8899f770544b420b7f/users/76d218362bc246f1bae64a0611d3e5a2/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:25.774739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:25.786365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a25e3a0b0944de7ad06005733861759'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:25.790375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:25.804524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a25e3a0b0944de7ad06005733861759'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:25.805443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a25e3a0b0944de7ad06005733861759'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:25.881909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-09e10f49-b13e-42ae-8ead-e543b1473882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a25e3a0b0944de7ad06005733861759'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:25.888966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec355a-bd6f-4c13-a213-e6da108d9d84 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:25.906084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:25.907206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.907586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.907953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] PATH_INFO: `/v3/projects/be61e4bb3db94e19b64534feb867b830` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.909609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=be61e4bb3db94e19b64534feb867b830)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:25.912376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:25.914880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1118/2234] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:25 2022] DELETE /identity/v3/users/9a25e3a0b0944de7ad06005733861759 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:25.916264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dacc3-ebe5-454f-b68c-de6d2c3678d3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:25.916561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dacc3-ebe5-454f-b68c-de6d2c3678d3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:25.916833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dacc3-ebe5-454f-b68c-de6d2c3678d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:25.962394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1a4dacc3-ebe5-454f-b68c-de6d2c3678d3 None None] MFA Rules not processed for user `76d218362bc246f1bae64a0611d3e5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:25.963750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1a4dacc3-ebe5-454f-b68c-de6d2c3678d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.047618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1119/2235] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:26.050106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:26.050673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:26.051182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:26.076754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be61e4bb3db94e19b64534feb867b830', 'request_id': 'req-0a2682d6-54bb-4539-9068-9e7580baa1de'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.080977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:26.082567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.083989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be61e4bb3db94e19b64534feb867b830', 'request_id': 'req-0a2682d6-54bb-4539-9068-9e7580baa1de'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.090119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be61e4bb3db94e19b64534feb867b830', 'request_id': 'req-0a2682d6-54bb-4539-9068-9e7580baa1de'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.091912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:26.147993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1120/2236] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:26.148861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:26.150729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.190562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:26.196977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] The token cache is being invalidate because project be61e4bb3db94e19b64534feb867b830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:26.197443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.198314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.200824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1117/2237] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:25 2022] DELETE /identity/v3/projects/be61e4bb3db94e19b64534feb867b830 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:26.201797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:26.203209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.223663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f085415-3a8d-449c-932d-d496f6fe2682 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:26.226969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:26.227631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:26.227911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:26.228186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:26.230065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.274436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2682d6-54bb-4539-9068-9e7580baa1de admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:26.281993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:26.283380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:26.283894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:26.284496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:26.287544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.304542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:26.306816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:26.310986 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:26.313623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:26.324102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1118/2238] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:26.325278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:26.327716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.332468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1121/2239] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:26.350371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5327c5d-6eb0-4e49-9e7e-e62ad4f24519 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:26.354559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:26.356000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:26.356510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:26.357029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] PATH_INFO: `/v3/users/5d6431ccbde541c48cbc3a9eb5275385` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:26.359265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6431ccbde541c48cbc3a9eb5275385)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:26.362462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:26.679194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6431ccbde541c48cbc3a9eb5275385'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.679929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6431ccbde541c48cbc3a9eb5275385'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.699513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6431ccbde541c48cbc3a9eb5275385'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.700973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6431ccbde541c48cbc3a9eb5275385'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.713960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ebccaaab-3bdf-4309-8e0f-3b1834f317e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6431ccbde541c48cbc3a9eb5275385'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:26.802717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1119/2240] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:26 2022] DELETE /identity/v3/users/5d6431ccbde541c48cbc3a9eb5275385 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:26.904706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:26.904706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:26.921398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ffc12-7d51-4430-841b-3d874fd4cce0 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.012548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.014010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.014392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.014676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] PATH_INFO: `/v3/projects/be45be35531949e4bb49f81dd456223b` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.015909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be45be35531949e4bb49f81dd456223b)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.017857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.031661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:27.105746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:27.159912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be45be35531949e4bb49f81dd456223b', 'request_id': 'req-74286267-58ea-474a-89b2-d408c855609a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.167162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be45be35531949e4bb49f81dd456223b', 'request_id': 'req-74286267-58ea-474a-89b2-d408c855609a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.173835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be45be35531949e4bb49f81dd456223b', 'request_id': 'req-74286267-58ea-474a-89b2-d408c855609a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.266621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] The token cache is being invalidate because project be45be35531949e4bb49f81dd456223b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:27.266972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.267637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-74286267-58ea-474a-89b2-d408c855609a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.269559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1122/2241] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:26 2022] DELETE /identity/v3/projects/be45be35531949e4bb49f81dd456223b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:27.275823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.277446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.277814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.302223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:27.305783 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.371212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1120/2242] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:27.381817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.382147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.382806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.404631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:27.406043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.416968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.420160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.423937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1123/2243] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:27.425186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.437215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c800a6-ca78-4d15-b26f-230bffbc676c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.446384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb0463-b1bd-4bb1-8c4e-9e0201cd1a12 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.449118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.450017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.450390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.450806 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.451598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:27.451910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.453574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.455785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.457499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.457499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.457499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] PATH_INFO: `/v3/projects/679a32c5db654974aaa65f6747f1134a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.458857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=679a32c5db654974aaa65f6747f1134a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.461861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.474910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1124/2244] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:38:27.476578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.491965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a6ed-9217-4c9d-b2e8-29de0c38045f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.500040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.501218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.501723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.502254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.503575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.505903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.590587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679a32c5db654974aaa65f6747f1134a', 'request_id': 'req-456ac8c6-98d5-4eb9-a104-d05de069f65d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.599896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679a32c5db654974aaa65f6747f1134a', 'request_id': 'req-456ac8c6-98d5-4eb9-a104-d05de069f65d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.608147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679a32c5db654974aaa65f6747f1134a', 'request_id': 'req-456ac8c6-98d5-4eb9-a104-d05de069f65d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.650132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1125/2245] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:27 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:27.650797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.652861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.724739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344395d-a97e-4af1-a249-bb7de38fa82d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.727501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] The token cache is being invalidate because project 679a32c5db654974aaa65f6747f1134a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:27.727728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.728406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.729954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.730105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1121/2246] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:27 2022] DELETE /identity/v3/projects/679a32c5db654974aaa65f6747f1134a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:27.730823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.732760 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.736167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.736696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.737181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] PATH_INFO: `/v3/users/4245fc4b5ad948898537af66820cca67` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.739322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4245fc4b5ad948898537af66820cca67)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.742855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.821361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ac8c6-98d5-4eb9-a104-d05de069f65d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:27.826525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:27.827205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.827487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.827761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.828460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:27.830647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc26cb0-d201-456d-a8a8-1fa68cac3890 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:27.870897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:27.938943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1122/2247] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:27 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:27.941277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:27.941873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:27.942481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:27.966360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:27.967737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:27.979260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1123/2248] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:27.986181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4245fc4b5ad948898537af66820cca67'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.987698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:27.994173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4245fc4b5ad948898537af66820cca67'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:27.999688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7aa3c3-f4cc-450a-b62a-25582771d349 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.006450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.007263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.007580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.007968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.008905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.010998 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584dbe2-e009-41c0-8616-bffa5c904a3b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.013633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4245fc4b5ad948898537af66820cca67'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:28.014962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4245fc4b5ad948898537af66820cca67'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:28.025734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1124/2249] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:28.027176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.027461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.027731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.043369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:28.052144 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:28.054578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.068770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1125/2250] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:28.070291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.088438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be0bd-9d3b-4a0e-aa0b-138801345a5f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.096586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.097750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.098309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.098801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.100178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.103391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.119234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1126/2251] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:28.120119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.132441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7a7d-2340-4ee5-a089-4aaba579b481 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.137958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.138756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.139160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.139513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.140475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:28.140843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.143324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.158368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1127/2252] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:28.159260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.170436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5160f-f549-46cb-9bcf-a127b523861e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.171343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4245fc4b5ad948898537af66820cca67'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:28.179352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.179352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.179352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.179352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] PATH_INFO: `/v3/projects/8ef7a1ee62504610a968e6211a67782a/users/f0f42e51693a433baee9e77001b8e6bb/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.182089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1126/2253] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:27 2022] DELETE /identity/v3/users/4245fc4b5ad948898537af66820cca67 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:28.183683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.188241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.198719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef7a1ee62504610a968e6211a67782a, user_id=f0f42e51693a433baee9e77001b8e6bb, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.202078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f74d6-fc81-40df-801a-6680b1b4a506 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.275417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1b5dc-a96d-4f5d-b2a3-dca6ca83f08a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.282402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.283569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.284079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.284585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.285946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.289763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469edb-8628-4d2e-ac89-7b1be27eca28 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.334432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1128/2254] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:28 2022] PUT /identity/v3/projects/8ef7a1ee62504610a968e6211a67782a/users/f0f42e51693a433baee9e77001b8e6bb/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:28.345014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.345501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.345980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.370712 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:28.372851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.394786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1127/2255] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:28 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:28.398739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.399597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.400079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.442949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] MFA Rules not processed for user `f0f42e51693a433baee9e77001b8e6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:28.444340 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.458255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1129/2256] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:28.459312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.470489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce7a4a-2165-4ef5-a051-d155217d3b55 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.486318 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:28.525233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1128/2257] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:28.526242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.527572 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.551952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:28.580114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.580114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.580578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.581080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.582421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.586865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.597948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3f2d1-a73f-4c28-820d-81d75a3065ad None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.602481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.603145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.603423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.603698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.617919 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.620964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.632676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1129/2258] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:28.633541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.634967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.718397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1130/2259] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:28 2022] POST /identity/v3/users => generated 287 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:28.719381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.738499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d477cf-dfe1-43ac-a68f-0ae8b9e5e745 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.744466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.745148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.745451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.745716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.746489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.749357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.754999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427258-fc79-437d-a9e9-cc5269666da0 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.758961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.759630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.759906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.760168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.768704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1131/2260] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:28.772745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.773345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.774989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.785552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1130/2261] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:28.790039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17238f10-6e10-4a9b-be87-3a1bbec732d3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.795256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.795950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.796406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.796893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.797973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.799898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.813797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1132/2262] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:28.818564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.831978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:28.834162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:28.834368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd179c6-8f31-418f-9fd9-14fd8d6ed53d service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.845086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.846665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.847135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.847624 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] PATH_INFO: `/v3/projects/313701a0b7184376946ba96a22c05ad1/users/c853e08893d741a9993289a223d2fbca/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.850580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4ff0-7f42-48d8-a0ee-dc0d2fe44415 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:28.857565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:28.858558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:28.858847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:28.859123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] PATH_INFO: `/v3/users/f0f42e51693a433baee9e77001b8e6bb` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:28.860493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f42e51693a433baee9e77001b8e6bb)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.862559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.880655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313701a0b7184376946ba96a22c05ad1, user_id=c853e08893d741a9993289a223d2fbca, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:28.886743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380d7b9-e912-4a95-b8e8-8f3a470ef1e0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:28.925786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:29.020563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1131/2263] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:28 2022] PUT /identity/v3/projects/313701a0b7184376946ba96a22c05ad1/users/c853e08893d741a9993289a223d2fbca/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:29.029473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.030755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.031280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.061782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:29.061782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.123499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1132/2264] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:29.133528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:29.175356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:29.177618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f42e51693a433baee9e77001b8e6bb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.178410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f42e51693a433baee9e77001b8e6bb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.193183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f42e51693a433baee9e77001b8e6bb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.194223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f42e51693a433baee9e77001b8e6bb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.203047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d0df29f0-bb1e-46ab-87d8-33ca1f969718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f42e51693a433baee9e77001b8e6bb'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.203800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a071324-c342-4c08-a3ef-79394ab430fd None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:29.207952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:29.208636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.208918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.209196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.211301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.224885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:29.226482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:29.238391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1133/2265] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:29.239324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:29.240693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.255171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551c1d-d45b-4bce-a79a-ceaf234a23a8 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:29.262991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:29.264086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.264628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.265016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] PATH_INFO: `/v3/projects/afe9e3392df044e5aa0c8627c179e807` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.267056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe9e3392df044e5aa0c8627c179e807)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:29.269247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:29.309022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1133/2266] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:28 2022] DELETE /identity/v3/users/f0f42e51693a433baee9e77001b8e6bb => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:29.314169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.314420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.314739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.333451 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:29.334776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.407744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1134/2267] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:29.422010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:29.425764 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.455994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe9e3392df044e5aa0c8627c179e807', 'request_id': 'req-a0a81852-2c22-452f-a64a-e112e8cc87af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.463902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe9e3392df044e5aa0c8627c179e807', 'request_id': 'req-a0a81852-2c22-452f-a64a-e112e8cc87af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.471231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe9e3392df044e5aa0c8627c179e807', 'request_id': 'req-a0a81852-2c22-452f-a64a-e112e8cc87af'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.496437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d08c-2529-4b6e-8f34-95338c0899a0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:29.558983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:29.560059 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.560325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.560616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.673960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:29.676691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:29.680909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] The token cache is being invalidate because project afe9e3392df044e5aa0c8627c179e807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:29.691335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.691335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:29.691335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1134/2268] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:29 2022] DELETE /identity/v3/projects/afe9e3392df044e5aa0c8627c179e807 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:29.697582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:29.701253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:29.726869 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1135/2269] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:29.762478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81852-2c22-452f-a64a-e112e8cc87af admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:29.782868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:29.786940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:29.789111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:29.789111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] PATH_INFO: `/v3/users/76d218362bc246f1bae64a0611d3e5a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:29.792118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d218362bc246f1bae64a0611d3e5a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:29.798188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:30.076199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d218362bc246f1bae64a0611d3e5a2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.093629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d218362bc246f1bae64a0611d3e5a2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.117648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d218362bc246f1bae64a0611d3e5a2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.118717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d218362bc246f1bae64a0611d3e5a2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.246648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d218362bc246f1bae64a0611d3e5a2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.254959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1135/2270] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:29 2022] DELETE /identity/v3/users/76d218362bc246f1bae64a0611d3e5a2 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:30.319370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:30.321520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:30.337642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f511a-6290-4002-8b2e-65fdb0fdb766 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:30.429839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:30.430780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:30.431063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:30.431338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] PATH_INFO: `/v3/projects/8ef7a1ee62504610a968e6211a67782a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:30.432686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef7a1ee62504610a968e6211a67782a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:30.435284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:30.589658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef7a1ee62504610a968e6211a67782a', 'request_id': 'req-f76e46bc-929f-488e-b63a-0a84bf54109d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.601828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef7a1ee62504610a968e6211a67782a', 'request_id': 'req-f76e46bc-929f-488e-b63a-0a84bf54109d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.613309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef7a1ee62504610a968e6211a67782a', 'request_id': 'req-f76e46bc-929f-488e-b63a-0a84bf54109d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.645562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:30.748774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] The token cache is being invalidate because project 8ef7a1ee62504610a968e6211a67782a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:30.749623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.751020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:30.756604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f76e46bc-929f-488e-b63a-0a84bf54109d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:30.761673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1136/2271] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:30 2022] DELETE /identity/v3/projects/8ef7a1ee62504610a968e6211a67782a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:31.014469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:31.024758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.050159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9873-df61-4b89-8913-e89b08278b14 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:31.124735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:31.125710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.125998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.126535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] PATH_INFO: `/v3/projects/2b44322704dd4d8899f770544b420b7f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.128172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b44322704dd4d8899f770544b420b7f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:31.130430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:31.160078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.160466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.160816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.193210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] MFA Rules not processed for user `c853e08893d741a9993289a223d2fbca`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:31.195041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.244000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:31.291111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1137/2272] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:31.304699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b44322704dd4d8899f770544b420b7f', 'request_id': 'req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:31.318694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b44322704dd4d8899f770544b420b7f', 'request_id': 'req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:31.328749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b44322704dd4d8899f770544b420b7f', 'request_id': 'req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:31.356005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:31.358729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.425851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd537b8-ccda-4e89-8241-442c38bd0892 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:31.467460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] The token cache is being invalidate because project 2b44322704dd4d8899f770544b420b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:31.467460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:31.467460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9c74570b-6dd2-4941-bdc0-dfd259f5db7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:31.472676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1136/2273] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:31 2022] DELETE /identity/v3/projects/2b44322704dd4d8899f770544b420b7f => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:31.476499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.476959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.477555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.502817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:31.504018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.507141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.507141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.508926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.513502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:31.515695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.599797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:31.601265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aa8b3-011b-463f-aefb-43631331444d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:31.612575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1138/2274] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:31.614608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1137/2275] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:31.622733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae167-667a-4810-9797-02e7c640022f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.623039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae167-667a-4810-9797-02e7c640022f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.623647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae167-667a-4810-9797-02e7c640022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.643499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b5fae167-667a-4810-9797-02e7c640022f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:31.644752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5fae167-667a-4810-9797-02e7c640022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:31.657327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1139/2276] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:31.662113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:31.677897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1eb8-a70c-4357-b6f9-37936312accc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:31.683535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:31.684205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.684485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.684831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.685547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:31.685855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:31.687556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:31.714704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1138/2277] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:38:31.717199 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fae167-667a-4810-9797-02e7c640022f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:31.729431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fae167-667a-4810-9797-02e7c640022f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:31.736690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:31.738290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.738572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.738905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.739654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:31.741740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:31.869646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1140/2278] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:31 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:31.874039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:31.884068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4a252-de0a-4741-bd37-cd509a08af9f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:31.890811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:31.949085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:31.949769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:31.950046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:31.951031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:31.952026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:31.953974 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.080015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1139/2279] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:31 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:32.084357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.099022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601b64-5b76-4059-9765-0a8f3939c057 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.108484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.110327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.111311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.111888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.113360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.115769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.129239 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1141/2280] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:32.134251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.151407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd902ba2-8c93-431f-b1a8-6064f039bae0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.157331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.158224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.158509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.158788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.159540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.161205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8823429-712a-494d-bdb5-0900c252cdf5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.173616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1140/2281] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:32.183721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.197692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f19c5-aff8-4ca7-a9a9-dc0522bdcdbf admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.205572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.208608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.208608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.208608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] PATH_INFO: `/v3/projects/13eec693d8e94980a6357730eb944284/users/93c53dd5423a434994905cced46e202b/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.234904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13eec693d8e94980a6357730eb944284, user_id=93c53dd5423a434994905cced46e202b, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.237864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.362627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1142/2282] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:32 2022] PUT /identity/v3/projects/13eec693d8e94980a6357730eb944284/users/93c53dd5423a434994905cced46e202b/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:32.368162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.368575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.369044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.419435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] MFA Rules not processed for user `93c53dd5423a434994905cced46e202b`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:32.420736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:32.518404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1141/2283] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:32.518624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.617055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc054a8-6999-4dd2-8344-b81dffbe173f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.622825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.623545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.623817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.627200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.627200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.627200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.638606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.640557 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:32.655891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510ed49-63db-4e65-81a4-b94cb9029b29 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.660558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.661469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.661737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.662007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] PATH_INFO: `/v3/users/9971ea7b401542e3b1a7efe3a897f64e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.663396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9971ea7b401542e3b1a7efe3a897f64e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.668588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.820609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1143/2284] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:32 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:32.824943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:32.906221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af8f11-2688-471c-950d-6f1a7b865682 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:32.914837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:32.918088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:32.918088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:32.918088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:32.918520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:32.922832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:32.924037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9971ea7b401542e3b1a7efe3a897f64e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:32.932052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9971ea7b401542e3b1a7efe3a897f64e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:32.945737 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9971ea7b401542e3b1a7efe3a897f64e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:32.946512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9971ea7b401542e3b1a7efe3a897f64e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:33.054115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1144/2285] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:32 2022] POST /identity/v3/users => generated 300 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:33.058678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.058968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ea2fd60e-02f0-44e9-a077-ed4103059ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9971ea7b401542e3b1a7efe3a897f64e'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:33.060737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.071258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1142/2286] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:32 2022] DELETE /identity/v3/users/9971ea7b401542e3b1a7efe3a897f64e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:33.074763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5d75-f55d-4042-93ec-d577c6aa22ac admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.075583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.076026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.076300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.099437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:33.101403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.152116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.152823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.153115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.153916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.154970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.156616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.175424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1143/2287] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:33.179148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:33.179148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1145/2288] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:33.179148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.185881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.187485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.197791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf79c0-9099-4ab2-89c5-d6fd96324b43 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.205130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.206182 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.206492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.206771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] PATH_INFO: `/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7/users/ceb75561779f440e83fd5c24a93ce8b9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.225087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359f0a4f732b4fc1a4fca38ec0b499d7, user_id=ceb75561779f440e83fd5c24a93ce8b9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.227901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.253060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41a7ca-2d77-4699-a050-28d910037f42 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.257281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.257928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.258374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.258669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.268421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.269867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.280020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1144/2289] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:33.351664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1146/2290] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:33 2022] PUT /identity/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7/users/ceb75561779f440e83fd5c24a93ce8b9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:33.356184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.367640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3203f47-d4c4-4b62-940c-77e49fe70f4e service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.438070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.438797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.439080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.439360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.440092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.441756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324a9c4-79f0-4770-9c7f-24f7ef968857 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.453160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1145/2291] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:33.457627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.469505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc41c23-4ecf-44da-adb4-47352a7d2c96 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.474993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.475924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.476238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.478940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] PATH_INFO: `/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7/users/ceb75561779f440e83fd5c24a93ce8b9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.478940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] RBAC: Authorizing `identity:create_grant(project_id=359f0a4f732b4fc1a4fca38ec0b499d7, user_id=ceb75561779f440e83fd5c24a93ce8b9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.481819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.493185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1147/2292] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:33 2022] PUT /identity/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7/users/ceb75561779f440e83fd5c24a93ce8b9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:33.512555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.513753 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.514039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.545307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] MFA Rules not processed for user `ceb75561779f440e83fd5c24a93ce8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:33.546599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.648057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1146/2293] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:33.666521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.676084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.802127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d936c-0757-4a8b-8131-0a4c1a27747c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.806921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.807741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.808018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.808264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.820302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.822064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.836915 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1148/2294] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:33.870878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.955936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fdd95-98f8-47b5-94d2-d074c2f1713a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:33.961168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:33.965379 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:33.977529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:33.977529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] PATH_INFO: `/v3/users/93c53dd5423a434994905cced46e202b` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:33.978160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:33.978415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c53dd5423a434994905cced46e202b)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:33.978504 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:33.978584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:33.996809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477ebb4-492f-41b6-bc9c-2420b41a1db6 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.002309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.003089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.003366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.003652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] PATH_INFO: `/v3/projects/dd8de9d9daad459d90f784a79173b759` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.004954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8de9d9daad459d90f784a79173b759)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.006945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.033043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:34.220223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c53dd5423a434994905cced46e202b'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.229363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8de9d9daad459d90f784a79173b759', 'request_id': 'req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.230496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c53dd5423a434994905cced46e202b'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.240606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8de9d9daad459d90f784a79173b759', 'request_id': 'req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.254090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c53dd5423a434994905cced46e202b'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.258155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8de9d9daad459d90f784a79173b759', 'request_id': 'req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.258385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c53dd5423a434994905cced46e202b'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.350610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-dceae944-8890-45a2-93ea-142316fbe1b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c53dd5423a434994905cced46e202b'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.357826 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1147/2295] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:33 2022] DELETE /identity/v3/users/93c53dd5423a434994905cced46e202b => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:34.363668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.363926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.364244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.387573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:34.387573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.447688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] The token cache is being invalidate because project dd8de9d9daad459d90f784a79173b759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:34.448064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.448757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d8dd2a25-a931-4ebd-96ec-22ec399c5d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:34.450987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1149/2296] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:33 2022] DELETE /identity/v3/projects/dd8de9d9daad459d90f784a79173b759 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:34.454771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1148/2297] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:34.456272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e9c1-ce7a-4080-8566-17adcf66d3ee None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.456573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e9c1-ce7a-4080-8566-17adcf66d3ee None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.456847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e9c1-ce7a-4080-8566-17adcf66d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.461651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:34.463355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.477671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7417e9c1-ce7a-4080-8566-17adcf66d3ee None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:34.480815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7417e9c1-ce7a-4080-8566-17adcf66d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.490659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1150/2298] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:34.500673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.502035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.502035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.523960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:34.525317 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.535183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1151/2299] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:34.540511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:34.545543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b14934-c95f-40a8-a93b-42e72285b1bb None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.551310 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.551982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.552260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.552530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.552831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afb6aa-00fb-4c9c-b3c7-afce01bfe604 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.554759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.558441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.559119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.559404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.562023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.562023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:34.562023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.563075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.569192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.570847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.578795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1152/2300] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:34.584030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1149/2301] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:34.585030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:34.594337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe96-be1e-4952-a586-9703ed9f0cea admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.599815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.600798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.601121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.602011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.602231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.605336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd07ed2-f27c-40b6-b294-f55c5d86cb17 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.725098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1153/2302] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:34.729234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:34.739532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bcf34-d5b7-438c-baa7-19f7af807ba0 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.748139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.748427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.749564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.770587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] MFA Rules not processed for user `c853e08893d741a9993289a223d2fbca`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:34.771973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:34.814168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.815359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.815359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.815600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.816361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.819324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.853383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1154/2303] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:34.939782 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1150/2304] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:34 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:34.945738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:34.956522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:34.968040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98812b4b-162c-483f-a9fc-9d77fd556505 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:34.974462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:34.975411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:34.975692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:34.976026 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:34.976779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:34.978581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:34.990983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1155/2305] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:34.995168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.010428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97680650-fa36-4296-ac61-1a6ffc197c76 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.021028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.023610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.023610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.023610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.023610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:35.027055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc52b2-0d44-413f-90c1-ba14d1a016fb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:35.054904 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.057350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:35.058480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1151/2306] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:35.069445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776b0cf-99f1-490d-b9e5-eb889b21746d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.081569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.082593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.082924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.083200 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] PATH_INFO: `/v3/projects/0a416245762e401abe03df82cba8b687/users/41f46104c88c49e88fe0909ae6e1b2d7/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.101680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a416245762e401abe03df82cba8b687, user_id=41f46104c88c49e88fe0909ae6e1b2d7, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:35.104589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:35.162402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1156/2307] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:35 2022] PUT /identity/v3/projects/0a416245762e401abe03df82cba8b687/users/41f46104c88c49e88fe0909ae6e1b2d7/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:35.174227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.174764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.175278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.224574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] MFA Rules not processed for user `41f46104c88c49e88fe0909ae6e1b2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:35.225567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.298853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.301541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.349739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1152/2308] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:35.349739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.349739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.407147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34ff36-4816-4431-89ed-83d2472bcbbc admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.412692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.413591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.413875 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.414539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] PATH_INFO: `/v3/users/00d7efeeb0cb44c5a4c7027b69d7286d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.415856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d7efeeb0cb44c5a4c7027b69d7286d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:35.418064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:35.435265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38f9b6-101e-4821-b070-c0a2f9d5040a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.440573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.441469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.441778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.442158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] PATH_INFO: `/v3/projects/13eec693d8e94980a6357730eb944284` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.444535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13eec693d8e94980a6357730eb944284)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:35.446562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:35.485195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:35.547500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:35.657161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d7efeeb0cb44c5a4c7027b69d7286d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.657959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d7efeeb0cb44c5a4c7027b69d7286d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.658786 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13eec693d8e94980a6357730eb944284', 'request_id': 'req-42ed5607-f693-4c6e-ab02-72121ebdae23'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.666335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13eec693d8e94980a6357730eb944284', 'request_id': 'req-42ed5607-f693-4c6e-ab02-72121ebdae23'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.673095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13eec693d8e94980a6357730eb944284', 'request_id': 'req-42ed5607-f693-4c6e-ab02-72121ebdae23'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.674095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d7efeeb0cb44c5a4c7027b69d7286d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.675130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d7efeeb0cb44c5a4c7027b69d7286d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.683159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d7efeeb0cb44c5a4c7027b69d7286d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.801858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] The token cache is being invalidate because project 13eec693d8e94980a6357730eb944284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:35.802929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.809065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:35.811473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1153/2309] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:35 2022] DELETE /identity/v3/projects/13eec693d8e94980a6357730eb944284 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:35.816373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.816373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.897854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5607-f693-4c6e-ab02-72121ebdae23 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.901975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.903019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.903941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.906070 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.910612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1157/2310] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:35 2022] DELETE /identity/v3/users/00d7efeeb0cb44c5a4c7027b69d7286d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:38:35.916606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:35.916606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.916827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:35.987902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2b652-4200-4403-b999-22037de80e1d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:35.993223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:35.994109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:35.994421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:35.994695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] PATH_INFO: `/v3/users/ceb75561779f440e83fd5c24a93ce8b9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:35.995927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb75561779f440e83fd5c24a93ce8b9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:35.998314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:36.001915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:36.001915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690319-d91e-453f-acd1-6c03011f5e3b service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:36.010652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1154/2311] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:36.012142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:36.012437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:36.012817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:36.038265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:36.042316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:36.053657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1155/2312] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:36.065912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:36.067566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:36.130603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f70d6d-7fe1-4654-8816-49e7f8e6e168 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:36.139989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:36.140771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:36.141071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:36.141421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:36.152161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:36.154244 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:36.165403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1156/2313] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:36.172317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:36.279503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb75561779f440e83fd5c24a93ce8b9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:36.280957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb75561779f440e83fd5c24a93ce8b9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:36.304754 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb75561779f440e83fd5c24a93ce8b9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:36.305475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb75561779f440e83fd5c24a93ce8b9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:36.314917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb75561779f440e83fd5c24a93ce8b9'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:36.436577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:36.507917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1158/2314] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:35 2022] DELETE /identity/v3/users/ceb75561779f440e83fd5c24a93ce8b9 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:36.773216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:36.869058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:36.871486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:36.888451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f8989-7337-4dbf-a7c7-1dadb5b9299e service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:36.996414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:36.998231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:36.998560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:36.998843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] PATH_INFO: `/v3/projects/0e6367d6524b4a999e92138487442177` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.000985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6367d6524b4a999e92138487442177)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:37.002970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:37.167405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6367d6524b4a999e92138487442177', 'request_id': 'req-90687737-3928-4301-9db8-da00aa021b9d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.176517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6367d6524b4a999e92138487442177', 'request_id': 'req-90687737-3928-4301-9db8-da00aa021b9d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.181966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:37.186863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:37.189784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:37.190973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6367d6524b4a999e92138487442177', 'request_id': 'req-90687737-3928-4301-9db8-da00aa021b9d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.208145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674c137-cce9-4572-b897-05a4aaa66086 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:37.274942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:37.275718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.275995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.276266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] PATH_INFO: `/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.277557 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] RBAC: Authorizing `identity:delete_project(project_id=359f0a4f732b4fc1a4fca38ec0b499d7)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:37.279719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:37.335359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] The token cache is being invalidate because project 0e6367d6524b4a999e92138487442177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:37.335643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.336273 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-90687737-3928-4301-9db8-da00aa021b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.337830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1157/2315] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:36 2022] DELETE /identity/v3/projects/0e6367d6524b4a999e92138487442177 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:37.457192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359f0a4f732b4fc1a4fca38ec0b499d7', 'request_id': 'req-7a992cca-6ead-477a-971e-dbaa40b5969f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.471705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359f0a4f732b4fc1a4fca38ec0b499d7', 'request_id': 'req-7a992cca-6ead-477a-971e-dbaa40b5969f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.484240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359f0a4f732b4fc1a4fca38ec0b499d7', 'request_id': 'req-7a992cca-6ead-477a-971e-dbaa40b5969f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.576609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] The token cache is being invalidate because project 359f0a4f732b4fc1a4fca38ec0b499d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:37.576609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.576609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7a992cca-6ead-477a-971e-dbaa40b5969f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:37.577908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1159/2316] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:37 2022] DELETE /identity/v3/projects/359f0a4f732b4fc1a4fca38ec0b499d7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:37.584867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.585353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.585727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.607728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:37.609000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:37.672842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1158/2317] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:37.681670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.682005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.682316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.703371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:37.704813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:37.717467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1160/2318] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:37.718956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:37.729769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bf2d9-4a21-4c60-9215-b5f54f45e435 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:37.735362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:37.736034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.736312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.736584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.737460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:37.737777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:37.739425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:37.753777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1159/2319] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:37.758447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:37.768728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac42a01-301f-44f5-b1fd-0d1c2c41784d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:37.773578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:37.774552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.775214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.775494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.776466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:37.782434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:37.831176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:37.871386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1161/2320] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:37 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:37.879209 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:37.881305 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:37.894001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c6d38-1382-4c32-b134-fb363b89d2a6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:37.981177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:37.984899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:37.984899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:37.984899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:37.984899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:37.985331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.094464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1160/2321] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:37 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:38.098681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.111262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635fc6-05ab-4af2-b384-dc3f54ab2baa admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.118326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.119065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.119346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.119615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.120422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.122081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.143941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1162/2322] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:38.146127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.173328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e2b5-cd98-486f-a796-dd714b9934bf admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.178689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.179383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.179670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.179945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.180681 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.182376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0232d35-d787-446f-a704-c8988050f0f1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.194319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1161/2323] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:38:38.198813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.209587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20b167-7835-4cca-9fa6-a91db540817c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.214099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.215010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.215289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.215558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] PATH_INFO: `/v3/projects/1beaf1b8aae046f99e5662b810d8b4ad/users/4a8be62c357543f28ed988d65e6dc2e4/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.232113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1beaf1b8aae046f99e5662b810d8b4ad, user_id=4a8be62c357543f28ed988d65e6dc2e4, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.235509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.248423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:38.336808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1163/2324] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:38 2022] PUT /identity/v3/projects/1beaf1b8aae046f99e5662b810d8b4ad/users/4a8be62c357543f28ed988d65e6dc2e4/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:38.350335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.350838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.351696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.386546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] MFA Rules not processed for user `4a8be62c357543f28ed988d65e6dc2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:38.388445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:38.457582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1162/2325] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:38.462243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.529935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872f96d-045f-4afe-9635-52d5f746295b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.535491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.536168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.536448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.536740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.537461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.539487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.556558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:38.667013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:38.681824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1164/2326] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:38 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:38:38.686649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.699578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc8c6-1928-444f-988a-b3e6a0a7318f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.806771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.807848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.809212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.809212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.811009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.814591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:38.953587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1163/2327] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:38 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:38.957681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:38.973360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2ce-b579-40cf-9aa8-ef5e278968cb admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:38.981983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:38.983427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:38.983746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:38.986376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:38.986376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:38.987854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:39.002795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1165/2328] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:39.006228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:39.018266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843ae2f-0739-4a7e-9f54-a5161c0f2579 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:39.028000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:39.029574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:39.029924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:39.030553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] PATH_INFO: `/v3/projects/ecabb13e408344e7b8cfa69b99e05935/users/030d7c2c92b54f1896b8a334db4da6f8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:39.049847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecabb13e408344e7b8cfa69b99e05935, user_id=030d7c2c92b54f1896b8a334db4da6f8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:39.053601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:39.163561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1164/2329] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:39 2022] PUT /identity/v3/projects/ecabb13e408344e7b8cfa69b99e05935/users/030d7c2c92b54f1896b8a334db4da6f8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:39.170745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:39.192019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7514-6101-4bd6-86ca-6d55f3f4a586 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:39.240725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:39.293099 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:39.294542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:39.294916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:39.295284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:39.296429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:39.298496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28942fb8-c69e-41be-ad80-18e94ddcae2d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:39.312626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1166/2330] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:39.318427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:39.330963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567c73-bc8f-40a9-af6b-4669f02f06f3 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:39.336833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:39.338168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:39.338474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:39.338836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] PATH_INFO: `/v3/projects/ecabb13e408344e7b8cfa69b99e05935/users/030d7c2c92b54f1896b8a334db4da6f8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:39.342262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecabb13e408344e7b8cfa69b99e05935, user_id=030d7c2c92b54f1896b8a334db4da6f8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:39.345181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:39.360562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1165/2331] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:39 2022] PUT /identity/v3/projects/ecabb13e408344e7b8cfa69b99e05935/users/030d7c2c92b54f1896b8a334db4da6f8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:39.372568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:39.373209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:39.373750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:39.423692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] MFA Rules not processed for user `030d7c2c92b54f1896b8a334db4da6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:39.425338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:39.460703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:39.504037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1167/2332] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:38:39.515433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:39.517205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:39.646433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605a805-363a-42e6-80ff-484b1a5a1e44 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:39.652240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:39.653248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:39.653392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:39.653674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:39.719752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:39.721175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb65416-d7a8-4cfb-b3e9-a12e016f203e service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:39.730879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1166/2333] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:38:40.004588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:40.337233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.003605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.112924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.147440 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.382584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.609675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.704944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:41.859025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:44.003165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:44.077675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9602d-51c7-4c23-8186-736cfa93c86b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:44.082672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:44.083592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:44.083873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:44.084149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] PATH_INFO: `/v3/users/4a8be62c357543f28ed988d65e6dc2e4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:44.085446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8be62c357543f28ed988d65e6dc2e4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:44.087493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:44.285770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8be62c357543f28ed988d65e6dc2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:44.286506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8be62c357543f28ed988d65e6dc2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:44.312401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8be62c357543f28ed988d65e6dc2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:44.313651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8be62c357543f28ed988d65e6dc2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:44.330552 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8be62c357543f28ed988d65e6dc2e4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:44.459817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1168/2334] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:44 2022] DELETE /identity/v3/users/4a8be62c357543f28ed988d65e6dc2e4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:44.465063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:44.465628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:44.465798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:44.489437 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:44.490766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:44.584099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1167/2335] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:44.595175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:44.596784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:44.642502 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:44.686434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c5e60-4502-4fa3-a1ad-22db98b5ac19 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:44.691632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:44.692297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:44.694681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:44.694681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:44.703996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:44.705725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:44.717380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1169/2336] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:45.273550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:45.276120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:45.292809 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87568f1d-a8b3-483a-8cf9-4ac369c2ea9f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:45.298102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:45.299210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:45.299568 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:45.300015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] PATH_INFO: `/v3/projects/1beaf1b8aae046f99e5662b810d8b4ad` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:45.303740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1beaf1b8aae046f99e5662b810d8b4ad)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:45.306024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:45.322461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:45.469118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1beaf1b8aae046f99e5662b810d8b4ad', 'request_id': 'req-efd2f28e-2412-4b4b-98b8-39171ae93023'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.479518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1beaf1b8aae046f99e5662b810d8b4ad', 'request_id': 'req-efd2f28e-2412-4b4b-98b8-39171ae93023'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.487159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1beaf1b8aae046f99e5662b810d8b4ad', 'request_id': 'req-efd2f28e-2412-4b4b-98b8-39171ae93023'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.555306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] The token cache is being invalidate because project 1beaf1b8aae046f99e5662b810d8b4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:45.556004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.557387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.560433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1168/2337] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:45 2022] DELETE /identity/v3/projects/1beaf1b8aae046f99e5662b810d8b4ad => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:45.566475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:45.582990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dd85b-1cb6-4cc0-aff6-e7882a4c4e05 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:45.671770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:45.672567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:45.672850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:45.673133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] PATH_INFO: `/v3/users/030d7c2c92b54f1896b8a334db4da6f8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:45.674461 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=030d7c2c92b54f1896b8a334db4da6f8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:45.676432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:45.925621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030d7c2c92b54f1896b8a334db4da6f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.926292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030d7c2c92b54f1896b8a334db4da6f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.941322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030d7c2c92b54f1896b8a334db4da6f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.942087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030d7c2c92b54f1896b8a334db4da6f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:45.951508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c326ba31-f17a-4d25-9621-3b39743f0a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030d7c2c92b54f1896b8a334db4da6f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.066600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1170/2338] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:38:45 2022] DELETE /identity/v3/users/030d7c2c92b54f1896b8a334db4da6f8 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:46.586263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:46.587940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:46.602760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2f28e-2412-4b4b-98b8-39171ae93023 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:46.672537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:46.673348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:46.673622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:46.673895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] PATH_INFO: `/v3/projects/ecabb13e408344e7b8cfa69b99e05935` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:46.675500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecabb13e408344e7b8cfa69b99e05935)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:46.677155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:46.846952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecabb13e408344e7b8cfa69b99e05935', 'request_id': 'req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.865659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecabb13e408344e7b8cfa69b99e05935', 'request_id': 'req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.875202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecabb13e408344e7b8cfa69b99e05935', 'request_id': 'req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.981427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] The token cache is being invalidate because project ecabb13e408344e7b8cfa69b99e05935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:38:46.981707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.982409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2e8f1062-320b-42f5-b5b8-c0aa1a058a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:38:46.984007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1169/2339] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:38:46 2022] DELETE /identity/v3/projects/ecabb13e408344e7b8cfa69b99e05935 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:46.989005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:46.989483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:46.989575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.008479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:47.009703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:47.072230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:47.096133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1171/2340] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:47.104767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.109752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.109752 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.125889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:47.127213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:47.138380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1170/2341] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:47.143176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.155611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94163d6-5718-434c-8948-5692e1027cf2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.161567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.162725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.162913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.163193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.163954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:38:47.164257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.166062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.189836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1172/2342] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:38:47.193956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.205375 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef072be-3f2b-44af-bd9e-108a767c5197 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.210663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.211509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.211791 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.212063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.212956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.215906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.423392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1171/2343] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:38:47 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:47.425009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.436983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd756fb4-126d-4a80-ad0a-938cc7563661 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.550690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.551483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.551839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.552801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.553000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.555842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.639968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1173/2344] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:38:47 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:38:47.644579 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.651748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:47.659258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad81f-2ca1-4039-9000-fd2636b6439d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.664531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.665227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.665506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.665793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.666583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.668213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.683003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1172/2345] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:47.687862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.702129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd548f89-550e-4a22-bd01-e0db79fdf45e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.707328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.708009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.708380 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.708873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.709629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.711392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d48a32-71a5-4677-b964-8caa0e5d8590 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.723062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1174/2346] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:38:47.729781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:47.741145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83987af-25fa-415d-b3f2-c93a54cb8725 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:47.748218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:47.749179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.749585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.749895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] PATH_INFO: `/v3/projects/bd95241bce3149fca8b80a04c77869b3/users/1b782eb6f7f5440aa561a3a65a97cb15/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.769122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd95241bce3149fca8b80a04c77869b3, user_id=1b782eb6f7f5440aa561a3a65a97cb15, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:47.771759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:47.898367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1173/2347] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:38:47 2022] PUT /identity/v3/projects/bd95241bce3149fca8b80a04c77869b3/users/1b782eb6f7f5440aa561a3a65a97cb15/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:38:47.912433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2715d21-b5fa-4ce7-a92b-d25d7c5510f7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:47.913587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2715d21-b5fa-4ce7-a92b-d25d7c5510f7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:47.914582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2715d21-b5fa-4ce7-a92b-d25d7c5510f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:47.963345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f2715d21-b5fa-4ce7-a92b-d25d7c5510f7 None None] MFA Rules not processed for user `1b782eb6f7f5440aa561a3a65a97cb15`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:38:47.964974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f2715d21-b5fa-4ce7-a92b-d25d7c5510f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:48.080402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1175/2348] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:38:48.098590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:38:48.100619 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:38:48.208554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d9ae9-e917-4265-a873-ef90fe15a57b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:38:48.212719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:38:48.213301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:38:48.213821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:38:48.214317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:38:48.225186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:38:48.226813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:38:48.238030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1174/2349] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:38:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:38:58.076659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:38:58.233465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:11.924951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad83f-8c5f-4bbe-bfb1-42815a60e44b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:11.925635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad83f-8c5f-4bbe-bfb1-42815a60e44b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:11.925635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ad83f-8c5f-4bbe-bfb1-42815a60e44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:11.951560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-fd3ad83f-8c5f-4bbe-bfb1-42815a60e44b None None] MFA Rules not processed for user `c853e08893d741a9993289a223d2fbca`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:11.957384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fd3ad83f-8c5f-4bbe-bfb1-42815a60e44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:12.047140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1176/2350] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:39:22.038376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=25 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:28.663115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:28.814187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ca983-ef40-44d8-9191-10eec9878844 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:28.822326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:28.823419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:28.823774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:28.824026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] PATH_INFO: `/v3/users/1b782eb6f7f5440aa561a3a65a97cb15` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:28.826307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b782eb6f7f5440aa561a3a65a97cb15)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:28.828691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:29.097022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b782eb6f7f5440aa561a3a65a97cb15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.107930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b782eb6f7f5440aa561a3a65a97cb15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.131208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b782eb6f7f5440aa561a3a65a97cb15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.132144 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b782eb6f7f5440aa561a3a65a97cb15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.203944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b782eb6f7f5440aa561a3a65a97cb15'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.215794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1175/2351] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:39:28 2022] DELETE /identity/v3/users/1b782eb6f7f5440aa561a3a65a97cb15 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:39:29.222173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:29.223039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:29.223039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:29.245592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:29.247930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:29.361251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1177/2352] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:39:29.377412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:29.380583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:29.511696 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:29.515014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:29.525197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549a11f-7828-468c-a920-4e44530b318e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:29.532713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:29.534041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:29.534602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:29.535134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:29.553084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:29.555203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:29.555565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c806-6e34-4d28-a08f-0bd41fc41e3c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:29.570343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:29.570343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:29.570343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:29.570343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] PATH_INFO: `/v3/users/41f46104c88c49e88fe0909ae6e1b2d7` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:29.571149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f46104c88c49e88fe0909ae6e1b2d7)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:29.573643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1176/2353] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:39:29.587128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:29.888984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:29.890667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:29.925563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f46104c88c49e88fe0909ae6e1b2d7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.927243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f46104c88c49e88fe0909ae6e1b2d7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.943198 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:29.956704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f46104c88c49e88fe0909ae6e1b2d7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.958541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f46104c88c49e88fe0909ae6e1b2d7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:29.985774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-803e0bdf-dc8e-414c-8522-9274508027ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f46104c88c49e88fe0909ae6e1b2d7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.139037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ad01b-075a-4958-b30b-c8b0884f52f6 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:30.169776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:30.171642 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.171931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.172531 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:30.181152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:30.188305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1178/2354] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:39:29 2022] DELETE /identity/v3/users/41f46104c88c49e88fe0909ae6e1b2d7 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:39:30.196016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.199862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.199862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:30.222951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:30.228775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:30.235977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:30.239302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:30.261376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1177/2355] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:39:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:39:30.353626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:30.356964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:30.383299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f866b-422a-4eea-a338-a110727a2568 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:30.389004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1179/2356] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:39:30.400910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:30.402715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.403488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.404095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] PATH_INFO: `/v3/projects/bd95241bce3149fca8b80a04c77869b3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:30.407136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd95241bce3149fca8b80a04c77869b3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:30.407332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:30.411951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:30.412236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:30.444372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:30.503470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b2e47-0d1b-408d-a7ca-8df753b5a903 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:30.509703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:30.510401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.510849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.511285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:30.524174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:30.526454 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e35917-d248-4d29-852d-09c0182e7461 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:30.552933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1180/2357] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:39:30.675596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd95241bce3149fca8b80a04c77869b3', 'request_id': 'req-f352f8cb-4623-462c-9f6f-56eb0b1e5816'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.687709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd95241bce3149fca8b80a04c77869b3', 'request_id': 'req-f352f8cb-4623-462c-9f6f-56eb0b1e5816'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.699648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd95241bce3149fca8b80a04c77869b3', 'request_id': 'req-f352f8cb-4623-462c-9f6f-56eb0b1e5816'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.837638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] The token cache is being invalidate because project bd95241bce3149fca8b80a04c77869b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:39:30.838057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.839221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-f352f8cb-4623-462c-9f6f-56eb0b1e5816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:30.842091 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1178/2358] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:39:30 2022] DELETE /identity/v3/projects/bd95241bce3149fca8b80a04c77869b3 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:39:30.848084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.848689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.849261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:30.885878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:30.887732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:30.979392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1181/2359] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:39:30.989047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:30.989708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:30.989708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.016846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:31.018317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:31.031372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1179/2360] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:39:31.036379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.048360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de85f8a-5a0b-4a58-8dcb-0c65bf5afdde None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.064636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.065334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.065617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.065962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.066771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:39:31.067084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.068752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.084903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1182/2361] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:39:31.091381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.343260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463a811-24da-4e9b-945a-1eace2490b0f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.343260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.343260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.343260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.345690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.345690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.345690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.460854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.462746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:31.476562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837f4db-e51d-4251-afaf-0b23250c3fba admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.482832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.483711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.483995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.484272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] PATH_INFO: `/v3/projects/0a416245762e401abe03df82cba8b687` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.484859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1180/2362] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:39:31 2022] POST /identity/v3/projects => generated 369 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:39:31.488312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a416245762e401abe03df82cba8b687)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.489060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.490937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.554801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:31.569527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a7cb-ea4b-465d-a5d2-892013353e8e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.574923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.575595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.575879 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.576158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.576915 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.578992 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.725097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a416245762e401abe03df82cba8b687', 'request_id': 'req-590ceb7f-ce56-443c-b34f-45c121cbded7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:31.735793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a416245762e401abe03df82cba8b687', 'request_id': 'req-590ceb7f-ce56-443c-b34f-45c121cbded7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:31.742014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1181/2363] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:39:31 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:39:31.745497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a416245762e401abe03df82cba8b687', 'request_id': 'req-590ceb7f-ce56-443c-b34f-45c121cbded7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:31.746391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.758795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ae31-7eaf-4a73-9286-d4dfcf08995d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.841246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.841987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.842420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.842747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.843668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.845363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.862096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] The token cache is being invalidate because project 0a416245762e401abe03df82cba8b687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:39:31.862566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:31.863067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1182/2364] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:39:31.865506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-590ceb7f-ce56-443c-b34f-45c121cbded7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:39:31.868256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1183/2365] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:39:31 2022] DELETE /identity/v3/projects/0a416245762e401abe03df82cba8b687 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:39:31.868629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.871941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:31.874474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c71b9-d2e2-456f-94ad-243210b9ecd0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.875042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c71b9-d2e2-456f-94ad-243210b9ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.876004 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c71b9-d2e2-456f-94ad-243210b9ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.898604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d81562-f224-4d26-81cf-a57db0d4a713 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.902772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-c81c71b9-d2e2-456f-94ad-243210b9ecd0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:31.905554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c81c71b9-d2e2-456f-94ad-243210b9ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:31.908802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.910125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.910644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.911124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.912481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.915623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:31.920974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1184/2366] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:39:31.930981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.931467 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.931940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.937043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1183/2367] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:39:31.941825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.959181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:31.968889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:31.969102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1b8c1-238c-4aa6-a6cd-e8d336136a6f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.971464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.972270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1185/2368] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:39:31.973191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.973716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.974225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] PATH_INFO: `/v3/projects/57a34ca04a424884b8bcec70eb693300/users/8e0f015ca3b840b5af855bbe9bd89fc8/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.976474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:31.987373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ee16-2b37-4b64-bada-89c8183291e7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:31.992491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:31.993238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:31.993529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:31.993732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:31.994538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:39:31.994839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:31.996548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.003459 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a34ca04a424884b8bcec70eb693300, user_id=8e0f015ca3b840b5af855bbe9bd89fc8, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.006762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:32.008923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b454037-862e-46ca-afa8-6f7d89dd58b7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.010633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1186/2369] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:39:32.016364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.029410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5400e7-2a10-4472-8b3d-bb21c28b6261 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.034691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.035421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.035789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.036135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.036883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.038972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.123446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1184/2370] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:39:31 2022] PUT /identity/v3/projects/57a34ca04a424884b8bcec70eb693300/users/8e0f015ca3b840b5af855bbe9bd89fc8/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:39:32.136783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.137218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.138186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.157368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1187/2371] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:39:32 2022] POST /identity/v3/projects => generated 391 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:39:32.164105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.184370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] MFA Rules not processed for user `8e0f015ca3b840b5af855bbe9bd89fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:32.187370 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:32.297245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20060b-00e0-4729-8bf3-09e5897cc578 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.299946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1185/2372] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:39:32.313921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.314448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.314968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.315423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.316677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.318890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.320868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:32.321752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.413719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:32.454279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1188/2373] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:39:32 2022] POST /identity/v3/users => generated 296 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:39:32.459066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.487147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3ff34-6307-472a-a7be-52a4634f9e88 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.495903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.499236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.499236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.499236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.501611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.504728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.509297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a843145-58ea-4b00-8839-3df7b61d0f91 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.514456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.515133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.515474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.515742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.526464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.528228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.534218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1189/2374] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:39:32.537143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.541242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1186/2375] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:39:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:39:32.583324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ebfa9-13d0-48d5-be1f-52486bf5a06c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.591926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.593196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.593784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.594403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.598610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.600347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541027ee-5b4a-4f9a-b3cb-9421bc153811 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.630281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1190/2376] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:39:32.632766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:32.668367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1dfe5-d2d8-49f5-8f79-b872326d07b2 service glance] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:32.690339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:32.691276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.695221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.695524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] PATH_INFO: `/v3/projects/c6a744cef1a9497cbcb855026ae3e7ce/users/7ada8373ee0348c48f5da84c1cc6e4a5/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.737372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a744cef1a9497cbcb855026ae3e7ce, user_id=7ada8373ee0348c48f5da84c1cc6e4a5, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:32.740348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:32.822438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1187/2377] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:39:32 2022] PUT /identity/v3/projects/c6a744cef1a9497cbcb855026ae3e7ce/users/7ada8373ee0348c48f5da84c1cc6e4a5/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:39:32.834670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac477ae-6c95-4d83-b95b-09d6b4c1e008 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:32.835005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac477ae-6c95-4d83-b95b-09d6b4c1e008 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:32.835212 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac477ae-6c95-4d83-b95b-09d6b4c1e008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:32.870539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dac477ae-6c95-4d83-b95b-09d6b4c1e008 None None] MFA Rules not processed for user `7ada8373ee0348c48f5da84c1cc6e4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:39:32.872481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dac477ae-6c95-4d83-b95b-09d6b4c1e008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:32.982457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1191/2378] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:39:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:39:33.645941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:39:33.648508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:39:33.720716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b2db-8abb-4c44-91d9-333ec406b3d0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:39:33.724936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:39:33.725757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:39:33.726037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:39:33.726416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:39:33.738155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:39:33.739698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7215b-6c15-4cd8-b109-0ecc621dcf04 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:39:33.751384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1188/2379] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:39:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:39:42.976125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=80,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:39:43.745492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:12.499616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:12.500529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:12.500529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:12.526133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] MFA Rules not processed for user `5132dcb5580d44e1bc2c7cf4a6893678`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:12.528650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:12.627873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1192/2380] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:14.420358 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:14.421362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:14.421362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:14.508132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:14.509467 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:14.554819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:14.663121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1189/2381] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:40:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:14.739493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:14.741260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:14.745211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2682c-5005-4cd7-b19c-ed2b05e38ba1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:14.754024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:14.755727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:14.756271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:14.756750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] PATH_INFO: `/v3/users/7ada8373ee0348c48f5da84c1cc6e4a5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:14.759018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ada8373ee0348c48f5da84c1cc6e4a5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:14.762361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:14.866960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac85d6f-e772-4209-913a-191dd420e9e5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:14.876353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:14.876353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:14.876727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:14.877023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:14.890507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:14.892851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f818fe-af4d-4880-b82e-d8ee2f9ab1db service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:14.908495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1190/2382] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:40:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:40:15.146537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ada8373ee0348c48f5da84c1cc6e4a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:15.147599 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ada8373ee0348c48f5da84c1cc6e4a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:15.183913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ada8373ee0348c48f5da84c1cc6e4a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:15.187446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ada8373ee0348c48f5da84c1cc6e4a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:15.207195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ada8373ee0348c48f5da84c1cc6e4a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:15.343370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1193/2383] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:40:14 2022] DELETE /identity/v3/users/7ada8373ee0348c48f5da84c1cc6e4a5 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:40:15.348844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:15.349715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:15.350304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:15.383175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:15.385871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:15.452088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:15.606240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1191/2384] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:15.612213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:15.635909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:15.671162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:15.755527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd08af-8d51-479a-875f-147577c104aa admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:15.761089 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:15.762680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:15.762969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:15.763245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:15.779737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:15.781540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1995f3-4482-46f1-aec0-de5c990d6cb9 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:15.802519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1194/2385] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:40:16.535035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:16.537314 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:16.565113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d620e0-d6db-46a5-84d3-1c1fb132ee0e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:16.580416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:16.583977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:16.584835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:16.585520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] PATH_INFO: `/v3/projects/c6a744cef1a9497cbcb855026ae3e7ce` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:16.588400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a744cef1a9497cbcb855026ae3e7ce)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:16.593662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:16.828295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a744cef1a9497cbcb855026ae3e7ce', 'request_id': 'req-847f59b0-65b3-4aed-a2da-22072a86b39c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:16.840045 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a744cef1a9497cbcb855026ae3e7ce', 'request_id': 'req-847f59b0-65b3-4aed-a2da-22072a86b39c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:16.851904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a744cef1a9497cbcb855026ae3e7ce', 'request_id': 'req-847f59b0-65b3-4aed-a2da-22072a86b39c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:16.974851 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] The token cache is being invalidate because project c6a744cef1a9497cbcb855026ae3e7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:40:16.976901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:16.978743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:16.983979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1192/2386] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:40:16 2022] DELETE /identity/v3/projects/c6a744cef1a9497cbcb855026ae3e7ce => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:40:26.143939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc862c3-c409-4dd1-bdd0-131ea3a296ee None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:26.149082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc862c3-c409-4dd1-bdd0-131ea3a296ee None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:26.149082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc862c3-c409-4dd1-bdd0-131ea3a296ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:26.264069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1bc862c3-c409-4dd1-bdd0-131ea3a296ee None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:26.265537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1bc862c3-c409-4dd1-bdd0-131ea3a296ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:26.372821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1195/2387] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:40:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:26.408470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:26.412912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:26.578185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f59b0-65b3-4aed-a2da-22072a86b39c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:26.587324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:26.588751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:26.590487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:26.591160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:26.605616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:26.608999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:26.641499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1193/2388] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:40:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:40:36.363414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=40 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:36.634564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:40.523973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:40.524510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:40.524510 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:40.543121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:40.544331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:40.553433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1196/2389] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:40:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:40.567077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:40.579961 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e23a-9db3-4da9-a704-784d1b163791 service swift] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:40.585129 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:40.585761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:40.586036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:40.586335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:40.594542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:40.596114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:40.605284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1194/2390] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:40:45.389101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:45.390596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:45.489724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82a92c-89b1-4f9d-ac00-554496367d82 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:45.495111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:45.496071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:45.496368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:45.496643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] PATH_INFO: `/v3/users/c853e08893d741a9993289a223d2fbca` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:45.497997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c853e08893d741a9993289a223d2fbca)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:45.500173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:45.877131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c853e08893d741a9993289a223d2fbca'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:45.888425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c853e08893d741a9993289a223d2fbca'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:45.897119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c853e08893d741a9993289a223d2fbca'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:45.898483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c853e08893d741a9993289a223d2fbca'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:45.906359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c853e08893d741a9993289a223d2fbca'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.050966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1197/2391] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:40:45 2022] DELETE /identity/v3/users/c853e08893d741a9993289a223d2fbca => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:40:46.351527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:46.371661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382070d9-ab6e-4182-be3a-063cd5ba9d6a service swift] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:46.503764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:46.504667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:46.504952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:46.505232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] PATH_INFO: `/v3/projects/313701a0b7184376946ba96a22c05ad1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:46.506522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313701a0b7184376946ba96a22c05ad1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:46.508463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:46.683107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313701a0b7184376946ba96a22c05ad1', 'request_id': 'req-e801552b-3798-42ac-9e9e-4f28bc0f3d07'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.689997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313701a0b7184376946ba96a22c05ad1', 'request_id': 'req-e801552b-3798-42ac-9e9e-4f28bc0f3d07'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.696297 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313701a0b7184376946ba96a22c05ad1', 'request_id': 'req-e801552b-3798-42ac-9e9e-4f28bc0f3d07'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.863684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] The token cache is being invalidate because project 313701a0b7184376946ba96a22c05ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:40:46.863684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.863684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e801552b-3798-42ac-9e9e-4f28bc0f3d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:46.863684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1195/2392] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:40:46 2022] DELETE /identity/v3/projects/313701a0b7184376946ba96a22c05ad1 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:40:47.842353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:47.843795 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:47.857165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff52-1bd3-4c14-8de8-1612ba5ab700 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:47.921270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:47.922338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:47.922658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:47.922980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] PATH_INFO: `/v3/users/8e0f015ca3b840b5af855bbe9bd89fc8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:47.924430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0f015ca3b840b5af855bbe9bd89fc8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:47.926727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:47.944710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:48.254672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0f015ca3b840b5af855bbe9bd89fc8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:48.255375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0f015ca3b840b5af855bbe9bd89fc8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:48.268705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0f015ca3b840b5af855bbe9bd89fc8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:48.269460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0f015ca3b840b5af855bbe9bd89fc8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:48.277958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0f015ca3b840b5af855bbe9bd89fc8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:48.410513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1198/2393] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:40:47 2022] DELETE /identity/v3/users/8e0f015ca3b840b5af855bbe9bd89fc8 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:40:48.415307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:48.415824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:48.415921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:48.433909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:48.435138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:48.496119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1196/2394] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:48.505785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:48.507259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:48.570017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70475b-497a-4931-bced-a7ffe97fa0f4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:48.574245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:48.574903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:48.575184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:48.575458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:48.585592 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:48.587067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852bd1f-111b-4650-a9cb-f3ba010d8048 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:48.598578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1199/2395] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:40:49.103414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.105336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.118509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4b161-94e9-45e4-ae30-59f90895a116 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:49.124274 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:49.125560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.125560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.125761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] PATH_INFO: `/v3/projects/57a34ca04a424884b8bcec70eb693300` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.126898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a34ca04a424884b8bcec70eb693300)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:49.128916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:49.141116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:49.278680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a34ca04a424884b8bcec70eb693300', 'request_id': 'req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:49.289640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a34ca04a424884b8bcec70eb693300', 'request_id': 'req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:49.298027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a34ca04a424884b8bcec70eb693300', 'request_id': 'req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:49.303928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fbcf-ae61-4b6b-9d81-d01ab5b26a47 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.304836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fbcf-ae61-4b6b-9d81-d01ab5b26a47 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.305103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fbcf-ae61-4b6b-9d81-d01ab5b26a47 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.306563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1200/2396] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:40:49.314093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.314725 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.314995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.380566 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:49.381886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.400478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] The token cache is being invalidate because project 57a34ca04a424884b8bcec70eb693300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:40:49.400837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:49.401878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-00e34bd5-95fe-4a6e-bd73-f18e18ef8d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:40:49.403646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1197/2397] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:40:49 2022] DELETE /identity/v3/projects/57a34ca04a424884b8bcec70eb693300 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:40:49.409404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478fa50-c472-4f7b-a411-80fa3799afa4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.409726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478fa50-c472-4f7b-a411-80fa3799afa4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.410057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478fa50-c472-4f7b-a411-80fa3799afa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.431206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4478fa50-c472-4f7b-a411-80fa3799afa4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:49.439363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4478fa50-c472-4f7b-a411-80fa3799afa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.460132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:49.470964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1201/2398] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:49.484193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.485748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.506399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1198/2399] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:49.509044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.509342 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.509734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.530267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:49.531560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.547055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12c11-81c7-4a63-b550-4a9d39cb853a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:49.548051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1199/2400] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:49.552466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.552811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:49.553473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.553755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.554031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.561725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118a7bb-eb20-463e-8b98-5f5cecce11e9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:49.564868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:49.566337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:49.570022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:49.576544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:49.577702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1202/2401] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:40:49.590242 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1200/2402] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:40:49.592213 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.602654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea317a-3a68-4bcc-b84b-abac9e23e7ac service glance] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:49.608370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:49.610934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.610934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.610934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.610934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:49.612164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa3bc2-806f-48de-8b8e-8eaecbeac571 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:49.849166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1203/2403] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/projects => generated 373 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:40:49.853407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.864000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e6e39-12a4-428e-8541-027d78720e37 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:49.892632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645f7e-a818-4fa9-814e-e77a5c881122 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.895389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645f7e-a818-4fa9-814e-e77a5c881122 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.895389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645f7e-a818-4fa9-814e-e77a5c881122 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.895389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1204/2404] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 09:40:49.899048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.899333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.899608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.918352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:49.919589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.923744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:49.924410 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:49.924761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:49.925036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:49.925918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:49.928203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:49.972408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1205/2405] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:49.984863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:49.986331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:49.996920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645db724-07bc-4ef9-8ff9-3785f7314216 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.000716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.001352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.001635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.001910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.009348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.011395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.019171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1206/2406] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:40:50.070842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1201/2407] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:40:49 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:40:50.072126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:50.084166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050eda6-f2ce-4221-b5f9-29747e0d3d7c service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.089539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.090209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.090496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.090769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.091486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.093080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc664-41dd-4b1b-8d45-46ce1cfbe858 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.104206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1207/2408] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:40:50.108580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:50.110291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c1da-f2cc-4057-a00a-c5fd00406d6a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.110749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c1da-f2cc-4057-a00a-c5fd00406d6a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.111069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c1da-f2cc-4057-a00a-c5fd00406d6a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.112412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1208/2409] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:40:50.118124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.118440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.118722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.118903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd1fb-d2e9-4d43-ae9f-1038447ae922 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.123829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.124669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.124830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.125306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.126972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.128621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.136515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:50.137494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:50.138744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:50.141513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1202/2410] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:40:50.153537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:50.163386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60589f2-111d-4f6f-b559-f5000ccb96c4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.168192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.169092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.169373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.169649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] PATH_INFO: `/v3/projects/2fb8d3ff68f0490d93f67d11141cd7b1/users/ff513010063c4dfcbce2e8572d7fd6c6/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.184970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb8d3ff68f0490d93f67d11141cd7b1, user_id=ff513010063c4dfcbce2e8572d7fd6c6, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.187699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.209490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1209/2411] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:40:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:50.220450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:50.229721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87949e7-5025-42f7-955b-8b6d18616ce7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.233356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.233976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.234309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.234622 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.246692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.248179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf20fc-0d38-4cf9-84d0-3f7ade03e04f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.259753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1210/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:40:50.305518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1203/2413] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:40:50 2022] PUT /identity/v3/projects/2fb8d3ff68f0490d93f67d11141cd7b1/users/ff513010063c4dfcbce2e8572d7fd6c6/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:40:50.317967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c3b-bc4e-423d-a573-de3efd756bfc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.318355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c3b-bc4e-423d-a573-de3efd756bfc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.318355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c110c3b-bc4e-423d-a573-de3efd756bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.347939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0c110c3b-bc4e-423d-a573-de3efd756bfc None None] MFA Rules not processed for user `ff513010063c4dfcbce2e8572d7fd6c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:50.349168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0c110c3b-bc4e-423d-a573-de3efd756bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:50.426261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1211/2414] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:40:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:40:50.429139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:40:50.438333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:50.440161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:50.517650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64c4b5-7751-4ccd-81ef-c018c6f02d1b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:50.521958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:50.522718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:50.522963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:50.523238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:50.532678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:50.544228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:50.545445 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1204/2415] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:40:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:40:54.944126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:54.944625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:54.944743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:54.963883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:40:54.965084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:55.025717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1212/2416] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:40:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:40:55.038103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:40:55.039571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:40:55.112855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acdf-018c-4c53-aedb-42b3786b7b37 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:40:55.118485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:40:55.119131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:40:55.119407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:40:55.119764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:40:55.129661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:40:55.131294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be3697-2eb6-4b6b-9225-f3067404957f service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:40:55.143591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1205/2417] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:41:05.018695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=67 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:05.139692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:07.141496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:07.203267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca93a6e-56cf-4216-8afb-7fe9a344f35b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:07.207883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:07.208771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:07.209052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:07.209327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] PATH_INFO: `/v3/users/ff513010063c4dfcbce2e8572d7fd6c6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:07.210534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff513010063c4dfcbce2e8572d7fd6c6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:07.212452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:07.392903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff513010063c4dfcbce2e8572d7fd6c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:07.393491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff513010063c4dfcbce2e8572d7fd6c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:07.406023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff513010063c4dfcbce2e8572d7fd6c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:07.406671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff513010063c4dfcbce2e8572d7fd6c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:07.413694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff513010063c4dfcbce2e8572d7fd6c6'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:07.501609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1213/2418] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:41:07 2022] DELETE /identity/v3/users/ff513010063c4dfcbce2e8572d7fd6c6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:41:07.507241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:07.507746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:07.507948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:07.525885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:07.527109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:07.588718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1206/2419] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:41:07.600265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:07.601694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:07.659575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0576-0670-489a-b7f7-7d2bab1d9c23 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:07.663111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:07.663734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:07.664016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:07.664298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:07.673942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:07.675402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e34a-e89c-4103-b19b-abaffd7d79e6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:07.685639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1214/2420] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:41:08.162431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.163878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:08.178799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f3eb3-2a64-417b-8351-251e9a2ab74d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.183966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.184829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.185113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.185388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] PATH_INFO: `/v3/projects/2fb8d3ff68f0490d93f67d11141cd7b1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.186729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb8d3ff68f0490d93f67d11141cd7b1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.188706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.310505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb8d3ff68f0490d93f67d11141cd7b1', 'request_id': 'req-d009504d-cb60-476b-9f8b-23bdc688db7a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:08.316587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb8d3ff68f0490d93f67d11141cd7b1', 'request_id': 'req-d009504d-cb60-476b-9f8b-23bdc688db7a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:08.321993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb8d3ff68f0490d93f67d11141cd7b1', 'request_id': 'req-d009504d-cb60-476b-9f8b-23bdc688db7a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:08.406623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] The token cache is being invalidate because project 2fb8d3ff68f0490d93f67d11141cd7b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:41:08.406929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:08.407548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d009504d-cb60-476b-9f8b-23bdc688db7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:08.409075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1207/2421] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:41:08 2022] DELETE /identity/v3/projects/2fb8d3ff68f0490d93f67d11141cd7b1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:41:08.413892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.414390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.414390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.432669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:08.433846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:08.450950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:08.494087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1215/2422] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:41:08.502054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.502371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.502649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.521239 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:08.522470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:08.530939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1208/2423] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:41:08.535085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.545261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bfbf4-758a-4ae7-b444-7ccc06aa5081 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.549926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.550621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.550902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.551442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.552158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:41:08.552463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.554088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.566683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1216/2424] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:41:08.570991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.580865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ccd2-cbaa-4333-8dc9-e948d679c80f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.585527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.586181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.586492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.586768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.587450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.589354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.697299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1209/2425] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:41:08 2022] POST /identity/v3/projects => generated 397 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:41:08.701713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.711819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e48168-6b32-40e8-a850-d5013861be47 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.768691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.769354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.769639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.769913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.770632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.772547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.851067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1217/2426] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:41:08 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:41:08.855878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.866441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1329c15-130e-4b29-afed-b89128f8882d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.871734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.872550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.872829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.873101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.873875 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.875529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.887813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1210/2427] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:41:08.892018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.903020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6adb32-f0b7-4ca2-8754-7e6517aa0cf6 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.907698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.908695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.908695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.909025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.909719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.911319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de66a0b-0a73-4c90-9bf1-7a2766223b94 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:08.925445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1218/2428] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:41:08.926660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:08.937766 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3afa2e-8750-4a24-9104-b6ae5ba25cb6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:08.942476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:08.943354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:08.943633 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:08.943904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] PATH_INFO: `/v3/projects/3109a2449ec7402595b174c04e97ef43/users/58a09d97961b48cb87eb988f993f5231/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:08.959640 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3109a2449ec7402595b174c04e97ef43, user_id=58a09d97961b48cb87eb988f993f5231, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:08.962352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:09.063613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1211/2429] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:41:08 2022] PUT /identity/v3/projects/3109a2449ec7402595b174c04e97ef43/users/58a09d97961b48cb87eb988f993f5231/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:41:09.076448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:09.076871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:09.077214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:09.103569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] MFA Rules not processed for user `58a09d97961b48cb87eb988f993f5231`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:09.104759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:09.168202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1219/2430] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:41:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:41:09.179852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:09.181503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:09.204248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:09.239760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce22d33-7de7-4a48-aef0-dcf1b6057e2a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:09.243444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:09.244076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:09.244357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:09.244637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:09.254614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:09.256049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c97e64-f0ec-49e0-ba83-22c7ab85d491 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:09.265388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1212/2431] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:41:19.165070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:19.417667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=17 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:21.941518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:22.012119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da74cb-038f-4f1d-aa25-1452a3184326 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:22.018039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:22.018929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:22.019210 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:22.019487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] PATH_INFO: `/v3/users/58a09d97961b48cb87eb988f993f5231` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:22.020672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a09d97961b48cb87eb988f993f5231)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:22.022621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:22.254428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a09d97961b48cb87eb988f993f5231'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:22.255157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a09d97961b48cb87eb988f993f5231'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:22.269926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a09d97961b48cb87eb988f993f5231'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:22.270845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a09d97961b48cb87eb988f993f5231'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:22.279859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a09d97961b48cb87eb988f993f5231'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:22.410091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1220/2432] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:41:21 2022] DELETE /identity/v3/users/58a09d97961b48cb87eb988f993f5231 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:41:22.415932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:22.416480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:22.416660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:22.438807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:22.440034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:22.510374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1213/2433] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:41:22.517378 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:22.518854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:22.596507 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd557cf-db4b-430b-959a-c11a844f642c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:22.600154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:22.600802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:22.601078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:22.601354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:22.612055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:22.613542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b776-6cd7-4073-b95a-aafecad5a989 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:22.623436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1221/2434] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:41:23.136323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:23.138994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:23.159489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9064e-678b-45f4-a5dc-e0361c0cc78f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:23.166955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:23.169436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:23.170214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:23.176269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] PATH_INFO: `/v3/projects/3109a2449ec7402595b174c04e97ef43` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:23.176269 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3109a2449ec7402595b174c04e97ef43)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:23.178861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:23.322179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3109a2449ec7402595b174c04e97ef43', 'request_id': 'req-503e6df1-84f8-4efd-830e-c41f1ebafed0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:23.329284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3109a2449ec7402595b174c04e97ef43', 'request_id': 'req-503e6df1-84f8-4efd-830e-c41f1ebafed0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:23.335931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3109a2449ec7402595b174c04e97ef43', 'request_id': 'req-503e6df1-84f8-4efd-830e-c41f1ebafed0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:23.518510 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] The token cache is being invalidate because project 3109a2449ec7402595b174c04e97ef43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:41:23.519208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:23.519419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:41:23.520957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1214/2435] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:41:23 2022] DELETE /identity/v3/projects/3109a2449ec7402595b174c04e97ef43 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:41:24.784446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:24.784918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:24.785012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:24.802485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:41:24.803680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:24.821048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:24.859668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1222/2436] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:41:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:41:24.882094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:24.883544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:24.954289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e6df1-84f8-4efd-830e-c41f1ebafed0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:24.959321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:24.959968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:24.960254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:24.960530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:24.968886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:24.970511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:24.980984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1215/2437] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:41:26.677076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:26.678733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:26.738077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2560e-387b-45ca-8089-84d75c72ef73 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:26.741967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:26.742654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:26.742934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:26.743201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:26.745079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:26.843229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:26.844628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:26.856512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1223/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:41:34.975615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:36.851164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=69 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:40.805632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:40.807106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:40.822675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04c39-b012-44c1-b53c-3d0e6718124e service swift] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:40.828667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:40.829309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:40.829592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:40.829870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:40.831669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:40.891492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:40.893542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:40.904102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1216/2439] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:41:41.037361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:41:41.039423 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:41.103841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9548ad-878f-4f95-801e-f1e4f73248f6 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:41:41.107604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:41:41.108267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:41:41.108554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:41:41.108827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:41:41.110731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:41:41.126022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:41:41.127513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:41:41.140450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1224/2440] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:41:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:41:50.899292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=28 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:41:51.134340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=26 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:42:01.394014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:01.395547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:01.408572 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9542e5-cec5-46e4-a49e-813e7fc08ce1 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:01.412221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:01.412972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:01.413259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:01.413537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:01.415367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:01.426628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:01.427997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:01.436455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1217/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:11.432636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:42:33.934105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:33.935619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:33.950346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af783e6b-6bc5-4d6a-b849-1b2f8e7af58d service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:33.956574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:33.957222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:33.957506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:33.957784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:33.960864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:34.020612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:34.022224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdddfa6-a7ca-4b88-a1ee-59f0af15b2c3 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:34.032357 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1225/2442] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:40.420785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:40.437216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515ecc7-b1b8-4821-85a3-f8c7118e9a72 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:40.445219 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:40.446294 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:40.446767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:40.447250 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:40.449910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:40.468903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:40.471106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b674c-9442-42ff-9e9e-6e27b7e29243 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:40.485746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1218/2443] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:44.027664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=28 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:42:45.349880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:45.354950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:45.354950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:45.368685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] MFA Rules not processed for user `5132dcb5580d44e1bc2c7cf4a6893678`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:42:45.369886 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:45.433720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1226/2444] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:42:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:42:46.175229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:46.175819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:46.175819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:46.204912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] MFA Rules not processed for user `8d52fe5018e2450dad79596fb1129c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:42:46.206189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:46.218426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1219/2445] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:42:46.712071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:46.722541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0fdb-369e-4b01-8735-56517b33de44 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:46.726120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:46.726799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:46.727081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:46.727353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:46.729633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:46.743928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:46.746142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:46.755536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1227/2446] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:48.191862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:48.193377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:48.267973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b36ea-8460-4289-85f0-0ce160ea266b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:48.273561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:48.274350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:48.274695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:48.275033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:48.277005 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:48.290059 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:48.291704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:48.309870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1220/2447] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:50.623534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:42:50.636565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ede70e-f3a2-47aa-9024-63a622324a35 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:42:50.642588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:42:50.643242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:42:50.643523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:42:50.643801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:42:50.645603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:42:50.657520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:42:50.659152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2ad32-7073-4a08-9b50-4915183e983f service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:42:50.668631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1228/2448] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:42:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:42:58.304621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=41 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:43:00.664253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=37 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:43:02.346783 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:43:02.362257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fec49-8d14-4fba-82bc-057671eeded7 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:43:02.371769 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:43:02.372801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:43:02.373265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:43:02.373703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:43:02.376774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:43:02.399997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:43:02.403231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:43:02.422661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1221/2449] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:43:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:43:12.415535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:43:20.383614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:43:20.384290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:43:20.384290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:43:20.402713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] MFA Rules not processed for user `e5081aed86004f6c99a05c9fbf2a36b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:43:20.403907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:43:20.412127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1229/2450] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:43:30.396730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:18.392371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:18.401862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206933f6-a68f-42a3-8c66-b198bb01215c service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:18.410464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:18.410464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:18.410464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:18.410464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:18.410842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:18.471758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:18.473187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:18.482946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1222/2451] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:22.503780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:22.508166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:22.530354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21124e47-4608-40cb-b290-1eeb9e3e0b4b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:22.538601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:22.539554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:22.539845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:22.540156 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:22.545736 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:22.569729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:22.571404 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:22.586457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1230/2452] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:22.705535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:22.707169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:22.725587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315955a-f158-4107-804e-3cc4af205a2a service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:22.728425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:22.729312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:22.729594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:22.729871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] PATH_INFO: `/v3/users/5132dcb5580d44e1bc2c7cf4a6893678` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:22.731107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5132dcb5580d44e1bc2c7cf4a6893678)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:22.733181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:22.942455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5132dcb5580d44e1bc2c7cf4a6893678'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:22.951502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5132dcb5580d44e1bc2c7cf4a6893678'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:22.967558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5132dcb5580d44e1bc2c7cf4a6893678'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:22.968660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5132dcb5580d44e1bc2c7cf4a6893678'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.105489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e12c1074-b50a-4847-8f60-a9692b608730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5132dcb5580d44e1bc2c7cf4a6893678'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.113247 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1223/2453] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:22 2022] DELETE /identity/v3/users/5132dcb5580d44e1bc2c7cf4a6893678 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:23.493457 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:23.508715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b6d19-3d9b-4b12-b523-919f459fcd54 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:23.581118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:23.582124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:23.582297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:23.583739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] PATH_INFO: `/v3/projects/91f6b3c676a84127989cc8e35ce91de1` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:23.585102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f6b3c676a84127989cc8e35ce91de1)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:23.587563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:23.740190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f6b3c676a84127989cc8e35ce91de1', 'request_id': 'req-2df4e28d-4d10-4a3e-93b1-feb49b837201'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.748602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f6b3c676a84127989cc8e35ce91de1', 'request_id': 'req-2df4e28d-4d10-4a3e-93b1-feb49b837201'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.755474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f6b3c676a84127989cc8e35ce91de1', 'request_id': 'req-2df4e28d-4d10-4a3e-93b1-feb49b837201'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.854839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] The token cache is being invalidate because project 91f6b3c676a84127989cc8e35ce91de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:23.855139 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.855857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-2df4e28d-4d10-4a3e-93b1-feb49b837201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:23.857401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1231/2454] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:23 2022] DELETE /identity/v3/projects/91f6b3c676a84127989cc8e35ce91de1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:23.863680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:23.864235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:23.864341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:23.897929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:23.904443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:23.969372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1224/2455] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:23.977481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:23.977769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:23.978078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.000327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:24.001546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:24.010128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1232/2456] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:44:24.016842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.029103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94684aa4-13e8-49d9-9c93-bafceaf26f35 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.034293 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.039958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.040541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:24.054444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1225/2457] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:44:24.059354 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.070083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdbc14-509c-4a64-bcd6-aa515bd48bf5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.075188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.075856 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.076136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.076409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.077124 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.079491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.237263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1233/2458] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:24 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:44:24.241722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.253560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bc72-a89b-4988-a813-7e600d0c969f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.314130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.314870 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.315177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.315560 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.316171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.318101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.489359 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1226/2459] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:24 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:24.493581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.504434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3ca70-f418-4af4-858f-59baefb51c7e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.510312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.512951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.512951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.512951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.512951 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.513876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.524559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1234/2460] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:24.528846 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.539923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52db88-1eb5-43aa-ab47-b6551948cf7b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.544580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.545235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.545508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.545778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.547154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.549258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988ed4-a086-425b-a242-53659f90c904 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.559973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1227/2461] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:24.564499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.576902 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f8ead-636f-4f8a-b2ac-023b98e84071 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.582985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.584034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.584382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.584704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] PATH_INFO: `/v3/projects/d186337ee2734c0ea0657dec972f18c7/users/054ea19a7eab4de89e86e4e12a48acf3/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.601215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d186337ee2734c0ea0657dec972f18c7, user_id=054ea19a7eab4de89e86e4e12a48acf3, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.604172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.682928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1235/2462] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:24 2022] PUT /identity/v3/projects/d186337ee2734c0ea0657dec972f18c7/users/054ea19a7eab4de89e86e4e12a48acf3/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:24.695048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.695048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.695802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.725793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] MFA Rules not processed for user `054ea19a7eab4de89e86e4e12a48acf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:24.727048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:24.796950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1228/2463] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:24.801923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:24.807683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:24.864453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37fbf-9b5b-4f63-a4a1-8de0d5ca89d0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:24.875535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:24.875535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:24.875535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:24.875535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:24.876294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:24.876294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:24.999797 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1236/2464] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:24 2022] POST /identity/v3/projects => generated 365 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:44:25.004064 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.015649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3efa3-b147-4ff6-aaa8-1e9b059f3d07 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.075462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.076042 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.076325 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.076679 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.078286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.080394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.088468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:25.211942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1229/2465] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:25 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:25.216025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.226604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cff9e-2798-448a-a30c-64971b56de28 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.231192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.231841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.232121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.232393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.233105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.234855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.245584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1237/2466] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:25.250571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.260947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf6785-23c7-4c34-8139-f5594d9f2a55 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.266421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.267316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.267595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.267863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] PATH_INFO: `/v3/projects/e1d80a85018749ae810266fa0863bd59/users/67607758ed294cb19ce9f4fa240cebe8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.285033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d80a85018749ae810266fa0863bd59, user_id=67607758ed294cb19ce9f4fa240cebe8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.287756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.393645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1230/2467] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:25 2022] PUT /identity/v3/projects/e1d80a85018749ae810266fa0863bd59/users/67607758ed294cb19ce9f4fa240cebe8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:25.398055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.408382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643e979-3ca4-4bc8-ade3-03095d241e10 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.469311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.470015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.470340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.470613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.471332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.472919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a4fb3-e17d-46cf-80b1-e29c2ff5ebc6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.484277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1238/2468] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:25.488565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.500990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba5288-130b-4622-9e84-39c61ef8bc70 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.505978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.506904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.507182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.507452 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] PATH_INFO: `/v3/projects/e1d80a85018749ae810266fa0863bd59/users/67607758ed294cb19ce9f4fa240cebe8/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.509573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d80a85018749ae810266fa0863bd59, user_id=67607758ed294cb19ce9f4fa240cebe8, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.512292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.523936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1231/2469] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:25 2022] PUT /identity/v3/projects/e1d80a85018749ae810266fa0863bd59/users/67607758ed294cb19ce9f4fa240cebe8/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:25.534405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.538405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.538405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.568611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] MFA Rules not processed for user `67607758ed294cb19ce9f4fa240cebe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:25.569876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:25.638611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:25.642612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1239/2470] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:25.657230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:25.659367 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:25.736159 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb3ae-366a-4825-9e89-2934456c77ab admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:25.740573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:25.741246 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:25.741528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:25.741856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:25.754587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:25.754804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:25.769376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1232/2471] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:26.031861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:26.043630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c26d-76d3-4eef-9912-d38fc2ebd1e5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:26.048362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:26.049379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:26.049713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:26.050039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:26.116627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:26.118286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:26.128253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1240/2472] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:32.523483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:32.525190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:32.610238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035d84-deaa-4155-85f4-63da24d603d4 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:32.615799 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:32.616020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:32.616302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:32.616602 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:32.618460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:32.631724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:32.633115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17030021-dc12-4871-b9fb-7fb9ef3bd7c7 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:32.645453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1233/2473] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:36.123627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=28 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:41.320670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:41.386891 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308005-df2a-4e59-b9db-abe16c929971 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:41.392079 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:41.392953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:41.393232 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:41.393508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] PATH_INFO: `/v3/users/054ea19a7eab4de89e86e4e12a48acf3` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:41.394787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] RBAC: Authorizing `identity:delete_user(user_id=054ea19a7eab4de89e86e4e12a48acf3)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:41.396896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:41.603183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054ea19a7eab4de89e86e4e12a48acf3'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:41.605446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054ea19a7eab4de89e86e4e12a48acf3'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:41.616864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054ea19a7eab4de89e86e4e12a48acf3'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:41.617595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054ea19a7eab4de89e86e4e12a48acf3'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:41.626270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054ea19a7eab4de89e86e4e12a48acf3'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:41.716983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1241/2474] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:41 2022] DELETE /identity/v3/users/054ea19a7eab4de89e86e4e12a48acf3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:41.721774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:41.722299 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:41.722398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:41.739721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:41.741027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:41.786329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:41.810146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1234/2475] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:41.819175 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:41.820629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:41.888289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77d18d-02de-4468-8bd7-47a584e8321c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:41.892170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:41.892812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:41.893088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:41.893361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:41.903180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:41.904614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:41.914673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1242/2476] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:42.419019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:42.420256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:42.434038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ddc5-1736-4102-bd2e-c20aebd8d70e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:42.438825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:42.439696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:42.440043 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:42.440298 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] PATH_INFO: `/v3/projects/d186337ee2734c0ea0657dec972f18c7` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:42.441538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d186337ee2734c0ea0657dec972f18c7)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:42.443571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:42.582116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd186337ee2734c0ea0657dec972f18c7', 'request_id': 'req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:42.588522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd186337ee2734c0ea0657dec972f18c7', 'request_id': 'req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:42.594763 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd186337ee2734c0ea0657dec972f18c7', 'request_id': 'req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:42.692576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] The token cache is being invalidate because project d186337ee2734c0ea0657dec972f18c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:42.692576 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:42.693203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:42.695775 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1235/2477] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:42 2022] DELETE /identity/v3/projects/d186337ee2734c0ea0657dec972f18c7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:42.699988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:42.712520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b9a-8f71-4c75-9c95-2ef3afd38286 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:42.739159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:42.785947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:42.786885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:42.787253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:42.787531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] PATH_INFO: `/v3/users/67607758ed294cb19ce9f4fa240cebe8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:42.788823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67607758ed294cb19ce9f4fa240cebe8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:42.790972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:43.025978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67607758ed294cb19ce9f4fa240cebe8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.026684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67607758ed294cb19ce9f4fa240cebe8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.039021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67607758ed294cb19ce9f4fa240cebe8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.039733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67607758ed294cb19ce9f4fa240cebe8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.046716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-49d82222-71b6-4a5d-afa4-b463c40a507a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67607758ed294cb19ce9f4fa240cebe8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.143909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1243/2478] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:42 2022] DELETE /identity/v3/users/67607758ed294cb19ce9f4fa240cebe8 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:43.644569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:43.646106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:43.660656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71cd8d-8f1e-4c77-aa94-1243ff8aa347 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:43.737463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:43.738424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:43.738707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:43.738983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] PATH_INFO: `/v3/projects/e1d80a85018749ae810266fa0863bd59` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:43.740360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d80a85018749ae810266fa0863bd59)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:43.742456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:43.895757 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d80a85018749ae810266fa0863bd59', 'request_id': 'req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.907232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d80a85018749ae810266fa0863bd59', 'request_id': 'req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:43.917419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d80a85018749ae810266fa0863bd59', 'request_id': 'req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:44.002363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:44.006853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] The token cache is being invalidate because project e1d80a85018749ae810266fa0863bd59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:44.007082 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:44.007742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e064daf4-9c6c-4568-8bfb-eeb4c16879c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:44.010853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1236/2479] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:43 2022] DELETE /identity/v3/projects/e1d80a85018749ae810266fa0863bd59 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:44.018517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.019300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.019300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.038808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:44.048183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:44.114281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1244/2480] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:44.122185 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.122489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.122767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.142530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:44.146963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:44.151832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1237/2481] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:44.156258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.166799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d63ca8-9c29-4782-9e56-d62522f95012 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.173654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.174372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.174648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.174965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.175871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:44:44.176216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.177854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.191046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1245/2482] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:44:44.195423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.205151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f39d-3e8f-4e7a-93e1-b43833679d6f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.209970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.210673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.210951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.211225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.211935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.213899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.333643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1238/2483] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:44.338853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.351403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ce64-b2de-4ac0-a664-2284ab9b3f20 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.390131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:44.421700 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.422308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.422593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.422867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.423905 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.425765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.565986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1246/2484] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:44.570137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.581945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1849f79-c179-4c49-b1d0-ab8de1eec9c6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.587079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.587743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.588020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.588289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.589204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.590917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.605202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1239/2485] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:44.610537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.621670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdaf48-52fc-451b-9506-cc9f3ba44954 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.626530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.627205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.627490 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.627763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.628493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.630106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d11c6a-b8f8-4a46-95c9-0f92579af838 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.642096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1247/2486] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:44:44.646862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:44.658738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba6e2c-560b-408d-94e5-2fe17e5bfc2d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:44.663645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:44.664528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.664805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.665078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] PATH_INFO: `/v3/projects/39870a1fd1ad4badbd89a2342f27ec5f/users/d2bce755f5da4abe84eb8b82825dda5e/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.682488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39870a1fd1ad4badbd89a2342f27ec5f, user_id=d2bce755f5da4abe84eb8b82825dda5e, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:44.687090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:44.781194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1240/2487] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:44 2022] PUT /identity/v3/projects/39870a1fd1ad4badbd89a2342f27ec5f/users/d2bce755f5da4abe84eb8b82825dda5e/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:44.794545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:44.795010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:44.795500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:44.846116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] MFA Rules not processed for user `d2bce755f5da4abe84eb8b82825dda5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:44.848371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:44.971811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1248/2488] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:44.976499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.051019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90580-c034-4bd6-8946-d00f72f7e611 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.056336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.057010 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.057287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.057558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.058284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.060265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:45.182670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1241/2489] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:44 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:45.188389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.210585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4157526-ada1-4d84-83b8-2b5788004473 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.289127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.289793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.290073 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.290383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.291080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.293018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:45.413329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1249/2490] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:45 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:45.417601 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.429567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a021-4ca4-4e27-8af3-44846093b585 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.435028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.435964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.436946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.437327 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.438264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.439991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:45.457006 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1242/2491] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:45.461273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.478478 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85f04a-ff87-4639-b17e-14c4882d552f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.485091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.486306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.486642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.486922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.488034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.489974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb98df7-fa66-4743-95fd-d594a5e13e2a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:45.501892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1250/2492] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:45.506994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.518035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c6c0d-7dbf-4e36-8e71-3b5b76db96d9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.523523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.523996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.524362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.524648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] PATH_INFO: `/v3/projects/19d7605a728e420699a394cd9fb6c1b8/users/bf141828c5c740a38e9e02f66c7f1121/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.542575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d7605a728e420699a394cd9fb6c1b8, user_id=bf141828c5c740a38e9e02f66c7f1121, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.545341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:45.675997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1243/2493] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:45 2022] PUT /identity/v3/projects/19d7605a728e420699a394cd9fb6c1b8/users/bf141828c5c740a38e9e02f66c7f1121/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:45.693813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.694235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.694511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.725381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] MFA Rules not processed for user `bf141828c5c740a38e9e02f66c7f1121`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:45.726676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:45.731157 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:45.805147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1251/2494] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:45.810522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:45.868876 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:45.884236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1db224-d6dd-4499-8ad5-a8d4388ac7c2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:45.888989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:45.889676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:45.889955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:45.890258 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:45.890976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:45.893137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.029855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1244/2495] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:45 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:46.034464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.045129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d670cc-63e3-4436-ad0e-5c16fee3f9af None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.111362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.112035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.112322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.112595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.113291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.115314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.301370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1252/2496] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:46 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:46.302021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.313136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247543c6-f11d-4bea-8fc8-082680a94bde admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.317831 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.318540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.318824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.319170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.319902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.321541 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.339131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1245/2497] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:46.350372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.360954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef1b5d-e102-4a2c-92b6-a70d6e0c3738 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.365747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.366713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.367040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.367319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] PATH_INFO: `/v3/projects/44b8a892e20d442895e27e1218e41381/users/971fd8c10bc1490abd48f643f4bab0a4/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.390963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b8a892e20d442895e27e1218e41381, user_id=971fd8c10bc1490abd48f643f4bab0a4, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.395005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.475184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1253/2498] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:46 2022] PUT /identity/v3/projects/44b8a892e20d442895e27e1218e41381/users/971fd8c10bc1490abd48f643f4bab0a4/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:46.479177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.489696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9c9c-a49f-4b42-a5a7-6ea8d30b3397 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.550287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.550949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.551224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.551547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.552212 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.553878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc13c6-1a16-4d24-b368-52f7a4c287cd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.564504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1246/2499] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:44:46.568744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.579444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5d714-0b00-468e-8577-250c174958a2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.584065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.584939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.585224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.585497 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] PATH_INFO: `/v3/projects/44b8a892e20d442895e27e1218e41381/users/971fd8c10bc1490abd48f643f4bab0a4/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.587508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b8a892e20d442895e27e1218e41381, user_id=971fd8c10bc1490abd48f643f4bab0a4, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.590190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.606738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1254/2500] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:46 2022] PUT /identity/v3/projects/44b8a892e20d442895e27e1218e41381/users/971fd8c10bc1490abd48f643f4bab0a4/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:46.619558 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.653178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.653178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.653178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] MFA Rules not processed for user `971fd8c10bc1490abd48f643f4bab0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:46.653178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:46.727090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1247/2501] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:46.740441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.742262 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:46.807489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2a60d-2e40-4bb3-9745-6a48bc05b65d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.807489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.807995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.808320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.808651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:46.818805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:46.820597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a031d-e337-4521-8b03-2a0119429fa0 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:46.828359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:46.833418 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1255/2502] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:46.971039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:46.972428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:46.984807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb89d9-7254-4123-a435-6a46ee599aeb None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:46.988821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:46.989468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:46.989748 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:46.990023 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:47.034390 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:47.054699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:47.056214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:47.066234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1248/2503] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:50.395069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:50.395656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:50.395824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:50.421228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:50.421228 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:50.480573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1256/2504] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:44:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:50.760511 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:50.762009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:50.835788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebee6b8-2457-4129-925b-bbbe5716d5b5 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:50.841496 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:50.842167 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:50.842508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:50.843309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:50.855123 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:50.856574 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d45532-60cc-4f5e-807b-ccb640da17eb service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:50.865812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1249/2505] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:44:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:44:53.095817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:53.175574 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ff2d-3ce4-4b25-a478-265aa9d51967 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:53.180715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:53.181714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:53.181994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:53.182303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] PATH_INFO: `/v3/users/d2bce755f5da4abe84eb8b82825dda5e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:53.183499 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2bce755f5da4abe84eb8b82825dda5e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:53.185477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:53.578234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2bce755f5da4abe84eb8b82825dda5e'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:53.579447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2bce755f5da4abe84eb8b82825dda5e'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:53.597986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2bce755f5da4abe84eb8b82825dda5e'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:53.598759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2bce755f5da4abe84eb8b82825dda5e'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:53.606676 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2bce755f5da4abe84eb8b82825dda5e'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:54.077812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1257/2506] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:53 2022] DELETE /identity/v3/users/d2bce755f5da4abe84eb8b82825dda5e => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:54.082474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:54.082969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:54.083066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:54.101492 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:54.102739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:54.164331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1250/2507] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:54.174823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:54.176257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:54.207435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:54.239187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a0601-feae-4271-853b-c6061a986516 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:54.242729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:54.243369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:54.243647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:54.244019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:54.257872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:54.259369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:54.273295 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1258/2508] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:44:54.866368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:54.868593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:54.907712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf2d85-939c-46e3-88bd-dc5266caa29a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:54.918236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:54.919256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:54.919614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:54.919798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] PATH_INFO: `/v3/projects/39870a1fd1ad4badbd89a2342f27ec5f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:54.923460 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39870a1fd1ad4badbd89a2342f27ec5f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:54.925469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:55.091234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39870a1fd1ad4badbd89a2342f27ec5f', 'request_id': 'req-03980bc6-7479-460c-847b-2e411641d3f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.100580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39870a1fd1ad4badbd89a2342f27ec5f', 'request_id': 'req-03980bc6-7479-460c-847b-2e411641d3f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.107422 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39870a1fd1ad4badbd89a2342f27ec5f', 'request_id': 'req-03980bc6-7479-460c-847b-2e411641d3f6'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.111636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:55.252130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] The token cache is being invalidate because project 39870a1fd1ad4badbd89a2342f27ec5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:55.252689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.253394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.257357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1251/2509] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:54 2022] DELETE /identity/v3/projects/39870a1fd1ad4badbd89a2342f27ec5f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:44:55.262352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:55.277761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0fd06-e456-41a6-bd2c-12c08ba91471 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:55.359882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:55.361306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:55.361781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:55.362050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] PATH_INFO: `/v3/users/bf141828c5c740a38e9e02f66c7f1121` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:55.363442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf141828c5c740a38e9e02f66c7f1121)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:55.365437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:55.702925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf141828c5c740a38e9e02f66c7f1121'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.704273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf141828c5c740a38e9e02f66c7f1121'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.722713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf141828c5c740a38e9e02f66c7f1121'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.724298 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf141828c5c740a38e9e02f66c7f1121'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.734806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf141828c5c740a38e9e02f66c7f1121'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:55.846284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1259/2510] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:55 2022] DELETE /identity/v3/users/bf141828c5c740a38e9e02f66c7f1121 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:56.450405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:56.454453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:56.482803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980bc6-7479-460c-847b-2e411641d3f6 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:56.583387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:56.584661 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:56.585179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:56.585709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] PATH_INFO: `/v3/projects/19d7605a728e420699a394cd9fb6c1b8` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:56.587264 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d7605a728e420699a394cd9fb6c1b8)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:56.589878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:56.781575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d7605a728e420699a394cd9fb6c1b8', 'request_id': 'req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:56.792556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d7605a728e420699a394cd9fb6c1b8', 'request_id': 'req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:56.804337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d7605a728e420699a394cd9fb6c1b8', 'request_id': 'req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:56.949173 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] The token cache is being invalidate because project 19d7605a728e420699a394cd9fb6c1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:56.949671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:56.951087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:56.959985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1252/2511] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:56 2022] DELETE /identity/v3/projects/19d7605a728e420699a394cd9fb6c1b8 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:56.964037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:56.965625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:56.980408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e93ff-ef7c-42e5-9d4d-3252f023dbf5 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:57.039153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:57.055653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:57.056347 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:57.056628 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:57.056966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] PATH_INFO: `/v3/users/971fd8c10bc1490abd48f643f4bab0a4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:57.058143 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=971fd8c10bc1490abd48f643f4bab0a4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:57.060113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:57.352503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971fd8c10bc1490abd48f643f4bab0a4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:57.353160 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971fd8c10bc1490abd48f643f4bab0a4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:57.380733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971fd8c10bc1490abd48f643f4bab0a4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:57.381532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971fd8c10bc1490abd48f643f4bab0a4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:57.394821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-4d0c96a2-61c8-418b-90ea-7048bb008925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971fd8c10bc1490abd48f643f4bab0a4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:57.899090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1260/2512] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:44:56 2022] DELETE /identity/v3/users/971fd8c10bc1490abd48f643f4bab0a4 => generated 0 bytes in 935 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:58.582949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:58.597007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:58.628080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d90d8-80ef-4f0d-9c36-06d0c8b40ad9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:58.715296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:58.716443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:58.716830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:58.717186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] PATH_INFO: `/v3/projects/44b8a892e20d442895e27e1218e41381` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:58.719303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b8a892e20d442895e27e1218e41381)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:58.721336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:58.925175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:58.945772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b8a892e20d442895e27e1218e41381', 'request_id': 'req-38ac19fd-1be7-4cf7-99c9-917d11d42924'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:58.953998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b8a892e20d442895e27e1218e41381', 'request_id': 'req-38ac19fd-1be7-4cf7-99c9-917d11d42924'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:58.961696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b8a892e20d442895e27e1218e41381', 'request_id': 'req-38ac19fd-1be7-4cf7-99c9-917d11d42924'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:59.131506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] The token cache is being invalidate because project 44b8a892e20d442895e27e1218e41381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:44:59.132871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:59.133573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-38ac19fd-1be7-4cf7-99c9-917d11d42924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:44:59.135581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1253/2513] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:44:58 2022] DELETE /identity/v3/projects/44b8a892e20d442895e27e1218e41381 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:44:59.142082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.142681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.142681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.162897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:59.164606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:59.234868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1261/2514] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:59.243702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.244238 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.244515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.269517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:44:59.270857 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:44:59.285441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1254/2515] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:44:59.287993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.299737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442f53b-64b5-492b-af92-e8ac3568e26f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.304755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.305517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.305889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.306363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.307253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:44:59.307567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.311815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:59.326842 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1262/2516] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:44:59.327838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.338180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93eabb-0b5f-4069-a3b3-6ee8988ae3a5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.346015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.347191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.347474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.347750 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.348444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.350385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:59.498655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1255/2517] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:44:59 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:59.500555 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.520246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deee70-6a45-4097-a38f-22031f523b90 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.587609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:44:59.598963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.599629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.599908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.600183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.600859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.602938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:59.769317 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1263/2518] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:44:59 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:44:59.781470 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.804332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4876b-3cc2-495f-892b-a967b4e36e5a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.812577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.814079 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.814780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.815577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.817113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.820922 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:59.842153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1256/2519] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:59.843920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.868229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aa279-2249-42a9-a00a-e756853f0a9d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.874851 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.875528 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.875952 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.876374 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.878291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.879926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bd05d-7295-424b-80ec-60223eec41c0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:44:59.900443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1264/2520] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:44:59.901962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:44:59.914001 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497c87-9816-409f-a547-1ec01ad628bf admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:44:59.919468 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:44:59.920472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:44:59.920822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:44:59.921104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] PATH_INFO: `/v3/projects/0d5f6fb2d63246be8528f25463ea475e/users/4245d2ae2162404f847da055c53e7ae9/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:44:59.943446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f6fb2d63246be8528f25463ea475e, user_id=4245d2ae2162404f847da055c53e7ae9, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:44:59.946542 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:00.092596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1257/2521] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:44:59 2022] PUT /identity/v3/projects/0d5f6fb2d63246be8528f25463ea475e/users/4245d2ae2162404f847da055c53e7ae9/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:00.107095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:00.107828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:00.108987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:00.142894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] MFA Rules not processed for user `4245d2ae2162404f847da055c53e7ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:00.144065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:00.224861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1265/2522] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:00.229504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:00.328807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d90478-8fae-48e8-a80a-70974a4a6c78 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:00.334948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:00.335723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:00.336007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:00.336280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:00.336980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:00.340343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:00.513304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1258/2523] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:45:00 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:00.521315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:00.535356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66da149-7000-4c63-8ffb-1b7d236f0cf1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:00.638369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:00.638852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:00.639732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:00.640060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:00.640753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:00.642733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:00.772580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:00.781501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1266/2524] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:45:00 2022] POST /identity/v3/users => generated 299 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:00.789734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f5fe-bfc4-4bd9-adcd-fc8435e4c1ef admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:00.795471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:00.796391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:00.796686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:00.797139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:00.797945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:00.799987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:00.815364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1259/2525] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:00.816748 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:00.832684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5adbf5-d477-46cf-b1e0-be83095e0c17 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:00.837651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:00.840947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:00.840947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:00.840947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] PATH_INFO: `/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5/users/77bb38e4a14e40adaca986942a83a799/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:00.876591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a31430abe145e3a5f1412ab2b4b0f5, user_id=77bb38e4a14e40adaca986942a83a799, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:00.882137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:01.009296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1267/2526] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:00 2022] PUT /identity/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5/users/77bb38e4a14e40adaca986942a83a799/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:01.013529 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:01.025575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bacac9-5064-45e9-ad02-5532337b2f7c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:01.052502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:01.114781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:01.115476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:01.115760 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:01.116033 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:01.116822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:01.118535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb11eb-5bb6-4e3f-98fc-6ff351186440 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:01.130927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1260/2527] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:01.135309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:01.147647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c9eaf-b776-4662-a240-7bcfcfd722a3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:01.152582 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:01.153527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:01.154019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:01.154397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] PATH_INFO: `/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5/users/77bb38e4a14e40adaca986942a83a799/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:01.156535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a31430abe145e3a5f1412ab2b4b0f5, user_id=77bb38e4a14e40adaca986942a83a799, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:01.159332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:01.171285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1268/2528] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:01 2022] PUT /identity/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5/users/77bb38e4a14e40adaca986942a83a799/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:45:01.185333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:01.185635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:01.185894 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:01.226778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] MFA Rules not processed for user `77bb38e4a14e40adaca986942a83a799`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:01.226778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:01.287736 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1261/2529] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:01.302460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:01.304444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:01.319534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:01.365243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bbe7-28c1-444b-8f66-ef6a180ff86a admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:01.368908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:01.369548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:01.369827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:01.370158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:01.427753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:01.429174 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb38066-da21-4a4e-a86c-81d559ba7337 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:01.438804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1269/2530] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:45:09.850160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:09.851728 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:09.864186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ed5ef-3418-4882-bc82-580546bf9447 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:09.868122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:09.868754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:09.869034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:09.869303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:09.879089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:09.880636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad0f45-0365-4e76-9e5f-e16b3c11ba11 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:09.891032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1262/2531] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:12.038985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.040613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.040613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.041077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1270/2532] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:45:12.047217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ecb2-7bd2-4888-9e4c-67c3b125070b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.047764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ecb2-7bd2-4888-9e4c-67c3b125070b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.047764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ecb2-7bd2-4888-9e4c-67c3b125070b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.067456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7420ecb2-7bd2-4888-9e4c-67c3b125070b None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:12.068695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7420ecb2-7bd2-4888-9e4c-67c3b125070b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:12.073695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=42 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:12.155303 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1263/2533] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:12.168608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:12.170608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:12.239564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cae3a4-d55a-49bc-9e35-8ac167f6cc28 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:12.243207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:12.243843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.244123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.244393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.255686 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:12.257125 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00ef5d-6649-492b-b4f5-d6cf35312a2e service glance] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:12.269214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1271/2534] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:12.654434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.654716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.655004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.656535 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1264/2535] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:45:12.661976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03086947-06ea-46a6-b8af-51e13957aab7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.662368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03086947-06ea-46a6-b8af-51e13957aab7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.662691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03086947-06ea-46a6-b8af-51e13957aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.682654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-03086947-06ea-46a6-b8af-51e13957aab7 None None] MFA Rules not processed for user `07600562586a47a696ec87ef60838559`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:12.683877 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-03086947-06ea-46a6-b8af-51e13957aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:12.738561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1272/2536] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:12.751353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:12.753072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:12.768836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda2793-4d71-46f2-9b5b-06dcc4b2c7f4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:12.772419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:12.773096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.773361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.773726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.781580 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:12.783065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561185c7-07bc-4128-9c76-30bf4d7e6cfb service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:12.791396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1265/2537] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:45:12.879776 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5cd7-db97-41d5-bdb9-49d0cc098ff8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.880095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5cd7-db97-41d5-bdb9-49d0cc098ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.880362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5cd7-db97-41d5-bdb9-49d0cc098ff8 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.881834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1273/2538] 10.222.0.48 () {54 vars in 921 bytes} [Fri May 27 09:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 09:45:12.887429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082039-80b1-4eb9-8013-f0c0a7a8a981 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.887715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082039-80b1-4eb9-8013-f0c0a7a8a981 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.887990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082039-80b1-4eb9-8013-f0c0a7a8a981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.906421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-03082039-80b1-4eb9-8013-f0c0a7a8a981 None None] MFA Rules not processed for user `55612ce26632413ebd0dfd5fc081b656`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:12.907605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-03082039-80b1-4eb9-8013-f0c0a7a8a981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:12.917031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1266/2539] 10.222.0.48 () {58 vars in 1018 bytes} [Fri May 27 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:12.928326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5cd7-db97-41d5-bdb9-49d0cc098ff8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:12.940558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5cd7-db97-41d5-bdb9-49d0cc098ff8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:12.944072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:12.944708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:12.944987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:12.945257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:12.957141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:12.958641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:12.969888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1274/2540] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:15.735309 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:15.735930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:15.735930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:15.744051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:15.755604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:15.756816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:15.821281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1267/2541] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:45:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:15.834008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:15.835889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:15.907941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405cae6-f3dc-413c-b0ca-3208564f3f0f service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:15.913179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:15.914526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:15.914808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:15.915081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:15.924898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:15.926556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44da8a0-e63c-4272-8bfc-53b78199453d service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:15.941263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1275/2542] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:45:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:45:17.916867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:18.012291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890a3e6-21c5-45b3-abbe-14d9eefad0ad None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:18.021777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:18.023621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:18.024292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:18.024907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] PATH_INFO: `/v3/users/4245d2ae2162404f847da055c53e7ae9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:18.027469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] RBAC: Authorizing `identity:delete_user(user_id=4245d2ae2162404f847da055c53e7ae9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:18.032245 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:18.414330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4245d2ae2162404f847da055c53e7ae9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:18.423092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4245d2ae2162404f847da055c53e7ae9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:18.443499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4245d2ae2162404f847da055c53e7ae9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:18.444623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4245d2ae2162404f847da055c53e7ae9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:18.641858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4245d2ae2162404f847da055c53e7ae9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:18.650816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1268/2543] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:45:17 2022] DELETE /identity/v3/users/4245d2ae2162404f847da055c53e7ae9 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:18.656087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:18.656972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:18.656972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:18.677941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:18.679411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:18.732269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:18.781900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1276/2544] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:18.794320 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:18.794634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:18.836503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:18.876373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701ce20-5a2e-461d-9b8e-df27c5a753df admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:18.881192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:18.882136 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:18.882839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:18.883428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:18.899065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:18.901076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:18.914015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1269/2545] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:19.847352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:19.849233 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:19.861976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f34ac-0bfa-4eee-89d8-55f68b6ca68a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:19.868062 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:19.868948 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:19.872184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:19.872184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] PATH_INFO: `/v3/projects/0d5f6fb2d63246be8528f25463ea475e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:19.872184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5f6fb2d63246be8528f25463ea475e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:19.882790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:20.058682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5f6fb2d63246be8528f25463ea475e', 'request_id': 'req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.066013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5f6fb2d63246be8528f25463ea475e', 'request_id': 'req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.072737 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5f6fb2d63246be8528f25463ea475e', 'request_id': 'req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.207531 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] The token cache is being invalidate because project 0d5f6fb2d63246be8528f25463ea475e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:45:20.207937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.208922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.210705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1277/2546] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:45:19 2022] DELETE /identity/v3/projects/0d5f6fb2d63246be8528f25463ea475e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:20.214905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:20.228881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749c2ff-d35d-4be9-8be4-fbc8262cd4a4 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:20.316450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:20.317363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:20.317636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:20.317905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] PATH_INFO: `/v3/users/77bb38e4a14e40adaca986942a83a799` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:20.319194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77bb38e4a14e40adaca986942a83a799)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:20.321177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:20.770942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77bb38e4a14e40adaca986942a83a799'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.785965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77bb38e4a14e40adaca986942a83a799'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.809673 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77bb38e4a14e40adaca986942a83a799'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.817291 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77bb38e4a14e40adaca986942a83a799'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.932734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-30c7e6f0-edd3-400f-b8eb-5bf9da5af31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77bb38e4a14e40adaca986942a83a799'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:20.945832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1270/2547] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:45:20 2022] DELETE /identity/v3/users/77bb38e4a14e40adaca986942a83a799 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:21.655028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:21.657521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:21.682241 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7adbd-fbdd-455c-b8ae-1315224c6f2f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:21.803041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:21.804343 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:21.804885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:21.805400 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] PATH_INFO: `/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:21.807086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a31430abe145e3a5f1412ab2b4b0f5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:21.809440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:22.056257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a31430abe145e3a5f1412ab2b4b0f5', 'request_id': 'req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:22.063691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a31430abe145e3a5f1412ab2b4b0f5', 'request_id': 'req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:22.065455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:22.070780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a31430abe145e3a5f1412ab2b4b0f5', 'request_id': 'req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:22.280172 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] The token cache is being invalidate because project a7a31430abe145e3a5f1412ab2b4b0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:45:22.280588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:22.281350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-d194279b-f1d9-4098-9f2a-d6cc17a88d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:22.283149 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1278/2548] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:45:21 2022] DELETE /identity/v3/projects/a7a31430abe145e3a5f1412ab2b4b0f5 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:22.289709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:22.293553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:22.293553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:22.310077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:22.311623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:22.365386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:22.393628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1271/2549] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:22.395437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:22.395755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:22.396033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:22.417707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:22.418962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:22.428055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1279/2550] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:22.432802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:22.443584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83541e00-7eb7-49b0-989a-25a2ce80dca8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:22.453427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:22.454092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:22.454538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:22.455040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:22.457804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:45:22.457804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:22.462110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:22.475135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1272/2551] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:45:22.479500 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:22.490042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57837812-0059-4f0e-b57f-6b7dd15406db None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:22.495489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:22.496493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:22.496493 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:22.497348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:22.498152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:22.500136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:22.877192 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1280/2552] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:45:22 2022] POST /identity/v3/projects => generated 385 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:22.877812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:22.888692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4f56-a366-4a07-87c7-0ac7843ad18b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:22.960994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:22.961764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:22.962112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:22.962417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:22.963181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:22.965438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:23.124955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1273/2553] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:45:22 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:23.126472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:23.142224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379435e5-ec84-4e54-acad-a6bc65e0731b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:23.147001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:23.147670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:23.147949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:23.148225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:23.148950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:23.150841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:23.171734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1281/2554] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:23.176257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:23.187416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d70c-6f7c-4368-8f6b-c60e80ea6c29 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:23.193142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:23.193829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:23.194112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:23.194571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:23.195311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:23.201525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded654e2-cd5f-4907-a22c-87457d1d3397 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:23.214439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1274/2555] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:23.217806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:23.244221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ca4a-9f79-4ad9-b90d-02d44fc27cd9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:23.251575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:23.252453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:23.252735 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:23.253015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] PATH_INFO: `/v3/projects/4c5811f5be924d9b964f875f883e9d5d/users/a06e47de381f4a0392bd430a51cbc818/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:23.276629 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5811f5be924d9b964f875f883e9d5d, user_id=a06e47de381f4a0392bd430a51cbc818, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:23.279530 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:23.433471 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1282/2556] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:23 2022] PUT /identity/v3/projects/4c5811f5be924d9b964f875f883e9d5d/users/a06e47de381f4a0392bd430a51cbc818/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:23.445747 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:23.446389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:23.446389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:23.486825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] MFA Rules not processed for user `a06e47de381f4a0392bd430a51cbc818`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:23.488989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:23.609962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1275/2557] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:23.618716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:23.699020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee6b1-05b9-42f2-b645-ecf673e8f022 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:23.699020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:23.699384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:23.699640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:23.699898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:23.700639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:23.702786 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:23.921892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1283/2558] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:45:23 2022] POST /identity/v3/projects => generated 385 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:45:23.926131 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:23.940878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb007-5ee9-4949-a300-4867efba520c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:23.989436 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:24.003109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.003907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.004384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.004855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.005643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.007900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:24.220793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1276/2559] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:45:23 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:24.224805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:24.235897 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c1771-00d4-47d7-84ea-ad81fa1cdd4d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:24.241049 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.241636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.241913 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.242184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.242934 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.244539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:24.257587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1284/2560] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:24.261169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:24.279418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8e3d-65f3-4d09-a365-bbaaf536a988 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:24.279418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.279418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.279418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.279794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.279794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.281189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a17eb9-7286-420a-9e64-c75231510aa3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:24.291877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1277/2561] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:24.296025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:24.306800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098c50a-72fb-482f-91d3-820399a792d4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:24.314074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.314615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.314900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.315303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] PATH_INFO: `/v3/projects/c4e53888bae042fbb4e36129882386bb/users/0ada2970489b4cc68d04e8764b7baafc/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.334690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e53888bae042fbb4e36129882386bb, user_id=0ada2970489b4cc68d04e8764b7baafc, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.337446 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:24.518578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1285/2562] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:24 2022] PUT /identity/v3/projects/c4e53888bae042fbb4e36129882386bb/users/0ada2970489b4cc68d04e8764b7baafc/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:24.534649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.534649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.534649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.561524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] MFA Rules not processed for user `0ada2970489b4cc68d04e8764b7baafc`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:24.562813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:24.631062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1278/2563] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:24.636791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:24.669115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:24.700766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dce4-2204-46c7-a8da-a8fc9b2c653b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:24.705694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.709468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.709468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.709468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.709468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.709848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:24.841963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1286/2564] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:45:24 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:24.846076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:24.857026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede3222-4ef9-45aa-91ec-7545a92ed112 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:24.916350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:24.917024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:24.917302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:24.917571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:24.918300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:24.920903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.122085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1279/2565] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:45:24 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:25.126030 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:25.136693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a6f6-15fd-481a-81d6-5af121ab9135 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:25.141281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:25.144643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.144643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.144643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.144643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:25.145038 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.156270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1287/2566] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:25.160674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:25.171963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410a9c6-dbb8-4230-8244-ef8291a2ef57 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:25.178003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:25.178929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.179208 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.179495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] PATH_INFO: `/v3/projects/5837cf4321634b2c94b571cc38fbda4e/users/75d95cf08e204299b475b420345a0463/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.196282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5837cf4321634b2c94b571cc38fbda4e, user_id=75d95cf08e204299b475b420345a0463, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:25.199013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.398240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1280/2567] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:25 2022] PUT /identity/v3/projects/5837cf4321634b2c94b571cc38fbda4e/users/75d95cf08e204299b475b420345a0463/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:25.403225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:25.415747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da874a-71d2-4d26-99d9-e144cc9df65f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:25.488829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:25.489594 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.490027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.490155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.490940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:25.492782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267a6b2-f532-4686-af0d-62cfd8e27888 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.504584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1288/2568] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:25.510365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:25.521836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bcf81-a5fa-4ba2-88b5-f6198e8c9cc0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:25.534028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:25.534028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.534028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.534028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] PATH_INFO: `/v3/projects/5837cf4321634b2c94b571cc38fbda4e/users/75d95cf08e204299b475b420345a0463/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.534553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5837cf4321634b2c94b571cc38fbda4e, user_id=75d95cf08e204299b475b420345a0463, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:25.534553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.546308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1281/2569] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:25 2022] PUT /identity/v3/projects/5837cf4321634b2c94b571cc38fbda4e/users/75d95cf08e204299b475b420345a0463/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:45:25.558240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.558694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.558978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.591449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] MFA Rules not processed for user `75d95cf08e204299b475b420345a0463`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:25.593214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:25.665142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1289/2570] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:25.682855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:25.686461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:25.703723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:25.760980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68f459-47a1-4351-85cd-980d06ae89de admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:25.765418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:25.766038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:25.766350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:25.766626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:25.846315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:25.847954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:25.858866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1282/2571] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:30.109023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:30.119051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef014a-e382-4bec-8693-2ed845493f9c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:30.122805 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:30.123519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:30.123829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:30.124096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:30.147816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:30.186816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:30.188240 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8541a-419f-405d-bb31-1df8c6087092 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:30.198179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1290/2572] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:32.412730 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:32.414117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:32.426073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef8db5-50f2-4ba5-822e-ad8f90ea2d81 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:32.429856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:32.430536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:32.430823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:32.431092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:32.441448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:32.442955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:32.453306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1283/2573] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:35.006369 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:35.006845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:35.006942 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:35.034188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:35.035465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:35.097767 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1291/2574] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:45:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:35.112842 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:35.114321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:35.188331 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2c00-cf65-422d-9588-58476658c222 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:35.193346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:35.194084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:35.194396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:35.201417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:35.204413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:35.206019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ea3fa-b00e-4fce-9ed4-de452debaa2c service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:35.215403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1284/2575] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:45:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:45:38.742970 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:38.821096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0f24-473a-465b-b33b-fd258c2f2ed4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:38.825793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:38.827236 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:38.827536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:38.827808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] PATH_INFO: `/v3/users/a06e47de381f4a0392bd430a51cbc818` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:38.829010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06e47de381f4a0392bd430a51cbc818)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:38.831050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:39.101097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06e47de381f4a0392bd430a51cbc818'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:39.101808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06e47de381f4a0392bd430a51cbc818'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:39.121204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06e47de381f4a0392bd430a51cbc818'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:39.121922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06e47de381f4a0392bd430a51cbc818'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:39.126364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:39.129075 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06e47de381f4a0392bd430a51cbc818'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:39.258919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1292/2576] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:45:38 2022] DELETE /identity/v3/users/a06e47de381f4a0392bd430a51cbc818 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:39.264268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:39.264807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:39.264807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:39.284339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:39.286272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:39.323404 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:39.351397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1285/2577] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:39.361409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:39.363397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:39.423395 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82953-693f-45b1-acec-4225a00e64f1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:39.427019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:39.427685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:39.427963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:39.428235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:39.437650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:39.439093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:39.449064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1293/2578] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:39.944024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:39.945448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:39.958621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa58cb-bac7-49c6-9ffb-6422e74e4d38 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:39.963523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:39.964392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:39.964672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:39.964944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] PATH_INFO: `/v3/projects/4c5811f5be924d9b964f875f883e9d5d` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:39.966130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5811f5be924d9b964f875f883e9d5d)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:39.969988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:40.103376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5811f5be924d9b964f875f883e9d5d', 'request_id': 'req-cedaaded-a888-408a-a952-cef694857194'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.110479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5811f5be924d9b964f875f883e9d5d', 'request_id': 'req-cedaaded-a888-408a-a952-cef694857194'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.116107 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5811f5be924d9b964f875f883e9d5d', 'request_id': 'req-cedaaded-a888-408a-a952-cef694857194'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.204164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] The token cache is being invalidate because project 4c5811f5be924d9b964f875f883e9d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:45:40.204385 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.204964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.206617 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1286/2579] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:45:39 2022] DELETE /identity/v3/projects/4c5811f5be924d9b964f875f883e9d5d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:40.210817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:40.221052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994186-978c-4cfc-817f-1e73ae63424d service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:40.276587 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:40.277441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:40.277715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:40.277978 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] PATH_INFO: `/v3/users/0ada2970489b4cc68d04e8764b7baafc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:40.279688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ada2970489b4cc68d04e8764b7baafc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:40.281607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:40.602314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ada2970489b4cc68d04e8764b7baafc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.603013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ada2970489b4cc68d04e8764b7baafc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.615359 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ada2970489b4cc68d04e8764b7baafc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.616179 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ada2970489b4cc68d04e8764b7baafc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.623148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ada2970489b4cc68d04e8764b7baafc'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:40.710102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1294/2580] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:45:40 2022] DELETE /identity/v3/users/0ada2970489b4cc68d04e8764b7baafc => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:41.178547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:41.179998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:41.194211 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaaded-a888-408a-a952-cef694857194 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:41.259287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:41.260281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:41.260695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:41.261013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] PATH_INFO: `/v3/projects/c4e53888bae042fbb4e36129882386bb` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:41.262866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e53888bae042fbb4e36129882386bb)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:41.264924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:41.414252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e53888bae042fbb4e36129882386bb', 'request_id': 'req-fd5e8039-b2f8-4961-a970-68bcd049da84'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.421550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e53888bae042fbb4e36129882386bb', 'request_id': 'req-fd5e8039-b2f8-4961-a970-68bcd049da84'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.427524 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e53888bae042fbb4e36129882386bb', 'request_id': 'req-fd5e8039-b2f8-4961-a970-68bcd049da84'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.429213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:41.599499 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] The token cache is being invalidate because project c4e53888bae042fbb4e36129882386bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:45:41.599958 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.600751 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.602442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1287/2581] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:45:41 2022] DELETE /identity/v3/projects/c4e53888bae042fbb4e36129882386bb => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:41.606481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:41.608496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:41.621806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c87b-fe9c-477c-b62f-e94c8f28d6c4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:41.627923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:41.684747 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:41.685570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:41.685849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:41.686126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] PATH_INFO: `/v3/users/75d95cf08e204299b475b420345a0463` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:41.687392 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d95cf08e204299b475b420345a0463)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:41.689367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:41.918269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d95cf08e204299b475b420345a0463'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.918926 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d95cf08e204299b475b420345a0463'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.931878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d95cf08e204299b475b420345a0463'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.932536 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d95cf08e204299b475b420345a0463'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:41.939789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1edec9c0-d33f-4f49-947f-1440901ffea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d95cf08e204299b475b420345a0463'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.073234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1295/2582] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:45:41 2022] DELETE /identity/v3/users/75d95cf08e204299b475b420345a0463 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:42.537503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:42.539041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:42.553279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e8039-b2f8-4961-a970-68bcd049da84 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:42.619192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:42.620076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:42.620353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:42.620688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] PATH_INFO: `/v3/projects/5837cf4321634b2c94b571cc38fbda4e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:42.621907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5837cf4321634b2c94b571cc38fbda4e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:42.624395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:42.782972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5837cf4321634b2c94b571cc38fbda4e', 'request_id': 'req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.790011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5837cf4321634b2c94b571cc38fbda4e', 'request_id': 'req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.796518 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5837cf4321634b2c94b571cc38fbda4e', 'request_id': 'req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.898218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] The token cache is being invalidate because project 5837cf4321634b2c94b571cc38fbda4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:45:42.898527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.899190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9d7bc485-cc91-46ba-86e3-a1c4cdf1ef89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:45:42.900824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1288/2583] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:45:42 2022] DELETE /identity/v3/projects/5837cf4321634b2c94b571cc38fbda4e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:42.906014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:42.906626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:42.906626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:42.928082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:42.929585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:42.994218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1296/2584] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:43.058140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.058386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.058670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.077004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:43.078241 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:43.088202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1289/2585] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:43.093960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.111244 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42a3e-3829-4559-9608-01955880d351 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.116426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.117000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.117280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.117554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.118327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:45:43.118636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.120237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.133854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1297/2586] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:45:43.138227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.147779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80388e7c-9cc1-4b42-800e-6c491f7a4c08 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.152724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.153234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.157683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.157683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.157683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.157683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.263761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1290/2587] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:45:43 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:45:43.269019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.280328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecddf-2087-49eb-a4c2-2066c485ac8c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.292665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:43.338597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.339123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.339405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.339682 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.340383 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.342981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.492779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1298/2588] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:45:43 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:45:43.496928 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.507292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11aa21-9d5d-44c1-a450-24232259a139 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.512179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.512835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.513141 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.513925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.514727 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.516341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.527037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1291/2589] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:43.531614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.541779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3983fa-2879-4a91-a074-5996547a39d0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.546918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.554255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.554255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.554255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.554255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.554255 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb0676-ec98-482d-b7a5-d094652e02ea admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.561784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1299/2590] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:45:43.565931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.576946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636ca32-9054-4827-93b5-d839fc5f29cb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.581564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.582514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.582793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.583062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] PATH_INFO: `/v3/projects/1c6cf7096e63465d8e861f9f82b21e2b/users/09e3e81754b345009e5d2b976ca321f4/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.601704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c6cf7096e63465d8e861f9f82b21e2b, user_id=09e3e81754b345009e5d2b976ca321f4, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.601902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.771044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1292/2591] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:45:43 2022] PUT /identity/v3/projects/1c6cf7096e63465d8e861f9f82b21e2b/users/09e3e81754b345009e5d2b976ca321f4/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:45:43.776299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a412-0a45-4633-8bc6-8b66f11c6d74 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.776624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a412-0a45-4633-8bc6-8b66f11c6d74 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.776919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a412-0a45-4633-8bc6-8b66f11c6d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.794122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a346a412-0a45-4633-8bc6-8b66f11c6d74 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:43.795364 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a346a412-0a45-4633-8bc6-8b66f11c6d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:43.859068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1300/2592] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:43.869278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:43.871073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:43.892378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:43.933859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad55321-5415-48f2-b1c0-091d60ced1f7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:43.938411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:43.939085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:43.939368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:43.939738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:43.949553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:43.951584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:43.965382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1293/2593] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:45.667874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d07c0-06cc-4e0c-b112-0ead72632b29 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:45.668408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d07c0-06cc-4e0c-b112-0ead72632b29 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:45.668520 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d07c0-06cc-4e0c-b112-0ead72632b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:45.696876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-276d07c0-06cc-4e0c-b112-0ead72632b29 None None] MFA Rules not processed for user `09e3e81754b345009e5d2b976ca321f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:45.698082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-276d07c0-06cc-4e0c-b112-0ead72632b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:45.764224 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1301/2594] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:45:45.774844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:45.776374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:45.833069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e305577-2f2f-4c90-bd6a-63121d07f048 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:45.836662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:45.837306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:45.837588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:45.837867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:45.846218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:45.847644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:45.856570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1294/2595] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:47.712519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:47.712519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:47.713458 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:47.734658 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] MFA Rules not processed for user `09e3e81754b345009e5d2b976ca321f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:45:47.735898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:47.743046 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1302/2596] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:45:48.004329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:48.005934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:48.027798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d1e2-249c-4a39-ad06-ead38942e89e service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:48.032738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:48.033506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:48.040231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:48.040231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:48.040231 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:48.048067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:48.049471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:48.058714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1295/2597] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:45:57.726297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:58.055225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=50 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:45:59.948195 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:45:59.958626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bd7a6-4b71-425e-a8db-71968ddc587e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:45:59.962516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:45:59.963173 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:45:59.963449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:45:59.963722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:45:59.965516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:45:59.978661 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:45:59.980105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:45:59.990691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1303/2598] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:04.041929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:04.045036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:04.059959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926fb3b-9116-47a3-bbcb-549051f8045c service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:04.064536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:04.065406 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:04.065689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:04.065968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] PATH_INFO: `/v3/users/e5081aed86004f6c99a05c9fbf2a36b9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:04.067196 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5081aed86004f6c99a05c9fbf2a36b9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:04.069114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:04.427362 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5081aed86004f6c99a05c9fbf2a36b9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:04.438691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5081aed86004f6c99a05c9fbf2a36b9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:04.452710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5081aed86004f6c99a05c9fbf2a36b9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:04.453483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5081aed86004f6c99a05c9fbf2a36b9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:04.641037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-c1316e0e-0652-4d7e-a853-a41094e82522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5081aed86004f6c99a05c9fbf2a36b9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:04.647584 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1296/2599] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:46:04 2022] DELETE /identity/v3/users/e5081aed86004f6c99a05c9fbf2a36b9 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:46:04.991165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:05.002132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95883212-0e91-42b3-beeb-dd61e4add448 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:05.063071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:05.063923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.064203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.064476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] PATH_INFO: `/v3/projects/64c171ba5fcb489d911aeb8c5abf4bb2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.065650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c171ba5fcb489d911aeb8c5abf4bb2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:05.067626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:05.302744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c171ba5fcb489d911aeb8c5abf4bb2', 'request_id': 'req-f43d7045-29d7-40c3-8d75-886b568ca5ef'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:05.309391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c171ba5fcb489d911aeb8c5abf4bb2', 'request_id': 'req-f43d7045-29d7-40c3-8d75-886b568ca5ef'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:05.316014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c171ba5fcb489d911aeb8c5abf4bb2', 'request_id': 'req-f43d7045-29d7-40c3-8d75-886b568ca5ef'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:05.470307 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] The token cache is being invalidate because project 64c171ba5fcb489d911aeb8c5abf4bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:46:05.470517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:05.471258 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-f43d7045-29d7-40c3-8d75-886b568ca5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:46:05.472873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1304/2600] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:46:04 2022] DELETE /identity/v3/projects/64c171ba5fcb489d911aeb8c5abf4bb2 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:46:05.478605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.479146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.479146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.501266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:46:05.502687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:05.566134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1297/2601] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:46:05.574134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.574463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.574743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.593436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:46:05.594664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:05.602580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1305/2602] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:46:05.607419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:05.617254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74725cca-b205-4918-87f9-0e6fce959544 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:05.621823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:05.622548 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.622828 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.623101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.623813 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:46:05.624115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:05.625700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:05.637312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1298/2603] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:46:05.641733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:05.651108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf34c5-098f-4ca4-9e55-3921368ae5f3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:05.656420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:05.656986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.657266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.657539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.658247 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:05.660148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:05.771577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1306/2604] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:46:05 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:46:05.773611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:05.785429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bf803-e655-4b0e-9657-a2e5ed269e2a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:05.850807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:05.851480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:05.851764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:05.852035 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:05.852723 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:05.854658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:05.861483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:05.998423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1299/2605] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:46:05 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:46:05.999535 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:06.013393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc22d1-f863-4ef2-8ffb-241d4edda1fe admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:06.015289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:06.016042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:06.016323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:06.016595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:06.017313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:06.019197 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:06.031165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1307/2606] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:46:06.035027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:06.045265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f461732-6572-48ab-b090-2c89fb26c834 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:06.058972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:06.058972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:06.058972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:06.058972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:06.059435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:06.059435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153b2c2-ab9f-4d31-b73d-23e3056cfb6c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:06.065316 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1300/2607] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:46:06.069744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:06.080953 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72a573-1c9e-4fd4-98bb-a58a26e9a652 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:06.085740 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:06.086755 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:06.087644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:06.087644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] PATH_INFO: `/v3/projects/1b2ae84b95ae44d18119450b94735f03/users/6772c708b48b404bb9f8794142ec6060/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:06.106956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2ae84b95ae44d18119450b94735f03, user_id=6772c708b48b404bb9f8794142ec6060, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:06.109685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:06.242916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1308/2608] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:46:06 2022] PUT /identity/v3/projects/1b2ae84b95ae44d18119450b94735f03/users/6772c708b48b404bb9f8794142ec6060/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:46:06.248984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093f8-df2b-4103-8771-ab37019e519a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:06.249267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093f8-df2b-4103-8771-ab37019e519a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:06.255934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093f8-df2b-4103-8771-ab37019e519a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:06.268415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-825093f8-df2b-4103-8771-ab37019e519a None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:46:06.269589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-825093f8-df2b-4103-8771-ab37019e519a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:06.336586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1301/2609] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:46:06.343290 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:06.345036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:06.361253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:06.416355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f788a-6e2b-4a20-a461-231bc176fc92 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:06.416355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:06.416355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:06.416355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:06.416818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:06.422903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:06.424340 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:06.434303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1309/2610] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:12.963200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f2a64-bcc6-49af-8003-52162fb4c31d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:12.963699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f2a64-bcc6-49af-8003-52162fb4c31d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:12.963921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f2a64-bcc6-49af-8003-52162fb4c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:12.994941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7c4f2a64-bcc6-49af-8003-52162fb4c31d None None] MFA Rules not processed for user `6772c708b48b404bb9f8794142ec6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:46:12.996307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7c4f2a64-bcc6-49af-8003-52162fb4c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:13.082651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1302/2611] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:46:13.098111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:13.099583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:13.187269 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16097a73-17f0-4599-857e-faeaf2922e10 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:13.193074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:13.193730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:13.194017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:13.194328 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:13.204061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:13.205804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:13.218525 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1310/2612] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:16.068217 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:16.068805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:16.068805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:16.092317 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] MFA Rules not processed for user `6772c708b48b404bb9f8794142ec6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:46:16.093636 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:16.101391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1303/2613] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:46:23.211791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=54 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:26.085027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:27.205018 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:27.215598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b05c-949c-4a02-b3f1-219cc7a15ab9 service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:27.219316 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:27.219959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:27.220237 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:27.220920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:27.222429 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:27.237242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:27.238790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:27.250961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1311/2614] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:34.526450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:34.528220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:34.544903 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577c62d-3561-49e6-a113-892afe5e5ba5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:34.550871 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:34.551521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:34.551804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:34.552068 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:34.554063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:34.568031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:34.569916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:34.582632 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1304/2615] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:37.245223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:43.390705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:43.392119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:43.404137 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05d35f-7cf4-4180-b458-72fde21ff818 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:43.408043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:43.408715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:43.409003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:43.409282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:43.411092 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:43.426939 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:43.428650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:43.440526 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1312/2616] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:44.577512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:53.435331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:46:56.230147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:56.239691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716a1-4615-41e3-81f4-8a366687b746 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:56.243450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:56.244074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:56.244389 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:56.244663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:56.246488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:56.258514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:56.259950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:46:56.271334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1305/2617] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:46:59.749122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:46:59.753302 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:46:59.765705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf837f8-afbe-4634-b797-1cd1c2b24434 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:46:59.770215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:46:59.771142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:46:59.771421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:46:59.771692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] PATH_INFO: `/v3/users/8d52fe5018e2450dad79596fb1129c64` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:46:59.786367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d52fe5018e2450dad79596fb1129c64)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:46:59.788339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:00.053718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d52fe5018e2450dad79596fb1129c64'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.054414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d52fe5018e2450dad79596fb1129c64'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.066360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d52fe5018e2450dad79596fb1129c64'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.067009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d52fe5018e2450dad79596fb1129c64'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.074009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-1b05f9ca-a55a-4841-9176-8dbde5a04f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d52fe5018e2450dad79596fb1129c64'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.160293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1313/2618] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:46:59 2022] DELETE /identity/v3/users/8d52fe5018e2450dad79596fb1129c64 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:00.413120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:00.423954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb95e5-2942-43a3-b641-7b7cc6dee988 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:00.484802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:00.485674 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:00.485956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:00.486260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] PATH_INFO: `/v3/projects/836f9646586d4f409c181cdbfa24cb32` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:00.499113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=836f9646586d4f409c181cdbfa24cb32)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:00.500536 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:00.673240 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836f9646586d4f409c181cdbfa24cb32', 'request_id': 'req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.680429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836f9646586d4f409c181cdbfa24cb32', 'request_id': 'req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.686236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836f9646586d4f409c181cdbfa24cb32', 'request_id': 'req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.934230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] The token cache is being invalidate because project 836f9646586d4f409c181cdbfa24cb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:00.934449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.935067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7e3a0d8b-4e48-4507-af4d-54788fb9a84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:00.936612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1306/2619] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:00 2022] DELETE /identity/v3/projects/836f9646586d4f409c181cdbfa24cb32 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:00.941726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:00.976449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:00.976449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:00.976449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:00.976449 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:01.021000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1314/2620] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:01.030176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.030500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.030784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.049718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:01.050959 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:01.058856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1307/2621] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:01.063060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.072685 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641bda1-8669-4f47-a423-271bab3aed00 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.077280 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.077898 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.078181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.078483 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.079281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:47:01.079590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.081180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.093463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1315/2622] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:47:01.097971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.108062 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06faca79-67f9-40d8-9b15-c8678294090e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.113098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.119395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.119395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.119395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.119395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.119395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.268803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1308/2623] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:01.273003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.283554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33727-d615-4366-961d-19ff197c0a3f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.327259 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:01.339633 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.340297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.340580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.340853 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.341586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.344187 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.460095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1316/2624] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:01.464493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.475656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9a505-12c7-4a1e-8b41-c95bf2c8f730 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.480911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.481612 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.481898 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.482182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.482936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.484545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.495514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1309/2625] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:01.499775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.510899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b2fd-7c97-4b4b-b68d-5b27ba3c202d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.516065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.516727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.517009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.517281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.518044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.519724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a7ce9-dd42-4cdd-b83e-1fbb3761d863 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.530815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1317/2626] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:01.534924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.545165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe6319-489c-4c7e-ace7-f47e49d10543 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.550220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.551104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.551378 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.551646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] PATH_INFO: `/v3/projects/d7eae2c6e34f457c899ff6fd37d8543a/users/276de0d2b05945d0850bc691da8b58bd/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.566778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7eae2c6e34f457c899ff6fd37d8543a, user_id=276de0d2b05945d0850bc691da8b58bd, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.569505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.658938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1310/2627] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:01 2022] PUT /identity/v3/projects/d7eae2c6e34f457c899ff6fd37d8543a/users/276de0d2b05945d0850bc691da8b58bd/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:01.669214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.669498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.669876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.697648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] MFA Rules not processed for user `276de0d2b05945d0850bc691da8b58bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:01.698874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:01.764527 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1318/2628] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:47:01.769902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.775811 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:01.831566 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e326d0b-d27b-4717-9b7e-37bd148d83fb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:01.836350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:01.837000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:01.837277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:01.837608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:01.838567 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:01.840477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:01.975936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1311/2629] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:01.979933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:01.990601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad8a3a-6c68-4f3c-a930-8f5892992559 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.046947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.047544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.047826 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.048102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.048789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.050787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.240547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1319/2630] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:01 2022] POST /identity/v3/users => generated 290 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:47:02.244513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:02.255500 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f81280-451d-4956-84e2-b9d54e1ffdd7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.260394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.261132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.261462 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.261807 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.262555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.264448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.275978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1312/2631] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:02.280541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:02.290779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ad368-30ae-44e0-8e75-57923e182955 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.295997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.297231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.297439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.297714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] PATH_INFO: `/v3/projects/bbc52be2493745fa9189447ba97c4630/users/8b48ef259c7048f7a2df7c0c6af02ee1/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.312859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc52be2493745fa9189447ba97c4630, user_id=8b48ef259c7048f7a2df7c0c6af02ee1, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.315556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.457492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1320/2632] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:02 2022] PUT /identity/v3/projects/bbc52be2493745fa9189447ba97c4630/users/8b48ef259c7048f7a2df7c0c6af02ee1/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:02.461591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:02.472126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd93f6-d502-4cf8-b1d4-68eec0805204 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.528968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.530121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.530423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.530694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.531399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.532978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a52938-131e-4659-a5f8-9490286767b3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.543106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1313/2633] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:02.548382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:02.558713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141b635-95cb-413d-b6da-85313f92c21d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.563260 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.564127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.564408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.564684 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] PATH_INFO: `/v3/projects/bbc52be2493745fa9189447ba97c4630/users/8b48ef259c7048f7a2df7c0c6af02ee1/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.566698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc52be2493745fa9189447ba97c4630, user_id=8b48ef259c7048f7a2df7c0c6af02ee1, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.569379 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.580203 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1321/2634] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:02 2022] PUT /identity/v3/projects/bbc52be2493745fa9189447ba97c4630/users/8b48ef259c7048f7a2df7c0c6af02ee1/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:02.590699 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.590988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.591265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.618012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] MFA Rules not processed for user `8b48ef259c7048f7a2df7c0c6af02ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:02.619234 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:02.685105 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:02.688715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1314/2635] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:02.699553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:02.701198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:02.712266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:02.758963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c002-f7e7-4cc5-bc09-7aa472a551a6 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:02.762845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:02.763987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:02.769095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:02.769095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:02.773597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:02.775045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93364e79-ef45-41f2-af85-a62ba6e0c2a1 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:02.784874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1322/2636] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:06.233203 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:06.298563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ee954-2c66-476c-bdb1-173a2675d279 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:06.303053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:06.303908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:06.304184 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:06.304453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] PATH_INFO: `/v3/users/276de0d2b05945d0850bc691da8b58bd` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:06.305850 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=276de0d2b05945d0850bc691da8b58bd)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:06.307936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:06.559837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276de0d2b05945d0850bc691da8b58bd'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:06.567792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276de0d2b05945d0850bc691da8b58bd'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:06.580606 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276de0d2b05945d0850bc691da8b58bd'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:06.581364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276de0d2b05945d0850bc691da8b58bd'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:06.672081 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276de0d2b05945d0850bc691da8b58bd'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:06.678537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1315/2637] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:06 2022] DELETE /identity/v3/users/276de0d2b05945d0850bc691da8b58bd => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:06.683133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:06.683612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:06.683612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:06.702617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:06.703995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:06.768769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1323/2638] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:06.778432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:06.779947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:06.841115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069c8f3-3c04-4d53-8f40-c8849e568f81 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:06.844793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:06.845432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:06.845709 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:06.845983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:06.855623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:06.857644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:06.868199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1316/2639] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:47:07.416705 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:07.418114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:07.430453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e793db-b1d0-49ba-aad9-c5602b2dc7ae None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:07.435900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:07.436762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:07.437067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:07.437338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] PATH_INFO: `/v3/projects/d7eae2c6e34f457c899ff6fd37d8543a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:07.438540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7eae2c6e34f457c899ff6fd37d8543a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:07.440450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:07.602549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7eae2c6e34f457c899ff6fd37d8543a', 'request_id': 'req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:07.609131 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7eae2c6e34f457c899ff6fd37d8543a', 'request_id': 'req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:07.615397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7eae2c6e34f457c899ff6fd37d8543a', 'request_id': 'req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:07.696183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] The token cache is being invalidate because project d7eae2c6e34f457c899ff6fd37d8543a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:07.696825 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:07.697082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:07.698823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1324/2640] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:07 2022] DELETE /identity/v3/projects/d7eae2c6e34f457c899ff6fd37d8543a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:07.703403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:07.713050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39689433-d9ce-4132-b38d-4f6a59d27a7e service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:07.731292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:07.768662 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:07.769578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:07.769858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:07.770133 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] PATH_INFO: `/v3/users/8b48ef259c7048f7a2df7c0c6af02ee1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:07.771357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b48ef259c7048f7a2df7c0c6af02ee1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:07.773270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:08.076854 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b48ef259c7048f7a2df7c0c6af02ee1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.083573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b48ef259c7048f7a2df7c0c6af02ee1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.095343 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b48ef259c7048f7a2df7c0c6af02ee1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.096019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b48ef259c7048f7a2df7c0c6af02ee1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.199374 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e92ce8b1-890b-4c4c-ac91-2c96588c0031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b48ef259c7048f7a2df7c0c6af02ee1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.206933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1317/2641] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:07 2022] DELETE /identity/v3/users/8b48ef259c7048f7a2df7c0c6af02ee1 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:08.684731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:08.686801 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:08.700164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bda12b-bfee-4b8d-8d19-ca5bc5effa06 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:08.737591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:08.762106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:08.763031 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:08.763314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:08.763588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] PATH_INFO: `/v3/projects/bbc52be2493745fa9189447ba97c4630` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:08.765455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc52be2493745fa9189447ba97c4630)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:08.767425 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:08.916749 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc52be2493745fa9189447ba97c4630', 'request_id': 'req-ada22d83-74a4-43c9-a744-dae44ef1559c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.923575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc52be2493745fa9189447ba97c4630', 'request_id': 'req-ada22d83-74a4-43c9-a744-dae44ef1559c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:08.929123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc52be2493745fa9189447ba97c4630', 'request_id': 'req-ada22d83-74a4-43c9-a744-dae44ef1559c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:09.016366 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] The token cache is being invalidate because project bbc52be2493745fa9189447ba97c4630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:09.016675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:09.017289 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-ada22d83-74a4-43c9-a744-dae44ef1559c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:09.018922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1325/2642] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:08 2022] DELETE /identity/v3/projects/bbc52be2493745fa9189447ba97c4630 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:09.023780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.024308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.024425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.042551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:09.043774 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:09.104083 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1318/2643] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:09.112524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.112810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.113116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.131660 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:09.132845 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:09.140653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1326/2644] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:09.144724 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.155490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ea19e-ace6-4a5c-aab1-90b418b56933 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.160039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.160707 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.160983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.161254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.161965 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:47:09.162295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.163891 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.176201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1319/2645] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:47:09.180323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.189508 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6375-4642-4bc6-b7e3-8e72ffd9fa0b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.194130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.194803 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.195082 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.195355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.196045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.197944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.287180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1327/2646] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:09.291372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.302333 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ac787-c9b2-4245-a622-788d82ee1d72 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.328954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:09.362140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.362863 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.363905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.363905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.364154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.365968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.490170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1320/2647] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/users => generated 304 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:09.494376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.505093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b92c8-1e11-475c-a620-dbf84338dd44 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.509916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.510625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.510909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.511185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.511914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.513540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.526245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1328/2648] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:09.531170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.546872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41d49-e52c-4117-ac6c-33853426a8f2 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.551862 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.552559 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.552829 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.553106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.553833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.555495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f77839-2279-4d29-85ed-1365fefa92c7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.569530 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1321/2649] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:47:09.570589 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.581407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf310a9-cf32-4d6c-b008-f3816be24200 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.586133 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.587053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.587332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.587607 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] PATH_INFO: `/v3/projects/5ae91e428f12454192d5fbfbdf0536db/users/01e9b0fa02c04198b53e877fb5400f8c/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.604267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae91e428f12454192d5fbfbdf0536db, user_id=01e9b0fa02c04198b53e877fb5400f8c, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.606992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.686165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1329/2650] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:09 2022] PUT /identity/v3/projects/5ae91e428f12454192d5fbfbdf0536db/users/01e9b0fa02c04198b53e877fb5400f8c/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:47:09.696794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.697076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.697348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.725498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] MFA Rules not processed for user `01e9b0fa02c04198b53e877fb5400f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:09.726860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:09.793329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1322/2651] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:09.798043 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.867515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52800f-4f94-43fe-a1dc-5d3625813424 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:09.872476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:09.873679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:09.873964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:09.874274 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:09.874982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:09.877001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:09.881647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:09.970561 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1330/2652] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/projects => generated 409 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:09.974458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:09.984971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614408a7-dc3c-4b0a-9da7-7ff9667bb993 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.042188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.042899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.043148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.043474 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.044109 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.046050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.163481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1323/2653] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:09 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:47:10.164974 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:10.176881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b123-e478-4adf-acf5-385b4d13f11f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.183324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.184070 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.184419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.184716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.186091 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.187782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.199538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1331/2654] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:47:10.203941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:10.214942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14a83-0ac1-4741-8d3d-b1c338858878 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.220977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.222051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.222722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.223061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] PATH_INFO: `/v3/projects/a6db376828b644dbbac754b1bc873417/users/8af478845fe04d88b1ac0e8b8360c56a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.241906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6db376828b644dbbac754b1bc873417, user_id=8af478845fe04d88b1ac0e8b8360c56a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.244838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.369418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1324/2655] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:10 2022] PUT /identity/v3/projects/a6db376828b644dbbac754b1bc873417/users/8af478845fe04d88b1ac0e8b8360c56a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:10.374014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:10.384140 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813a0f3-f175-443e-baba-843fac908d7c admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.443466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.444152 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.444430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.444701 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.445420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.447032 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a810a-b5d0-4d78-a8dd-8a956417716d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.457341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1332/2656] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:10.464957 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:10.466861 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:10.472371 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132fb2a-5dc4-48fb-9f8a-a2f90089237f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.477065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.477931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.478237 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.478522 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] PATH_INFO: `/v3/projects/a6db376828b644dbbac754b1bc873417/users/8af478845fe04d88b1ac0e8b8360c56a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.480476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6db376828b644dbbac754b1bc873417, user_id=8af478845fe04d88b1ac0e8b8360c56a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.483725 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.493812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1325/2657] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:10 2022] PUT /identity/v3/projects/a6db376828b644dbbac754b1bc873417/users/8af478845fe04d88b1ac0e8b8360c56a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:10.505487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.505775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.511643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.533603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] MFA Rules not processed for user `8af478845fe04d88b1ac0e8b8360c56a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:10.534802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:10.609874 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1333/2658] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:10 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:10.620700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:10.622332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:10.680357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0611a2d-1061-400b-9fa9-c67046bde069 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:10.684004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:10.684645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:10.684921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:10.685191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:10.694716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:10.696148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85135f4-21dc-4eef-ad70-b4c99f65f09c service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:10.705932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1326/2659] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:11.226442 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:11.237698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94acb7f-5e87-4ad3-bc81-ef71a2e82efa None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:11.241390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:11.242063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:11.242386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:11.242662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:11.302512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:11.305058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:11.306540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:11.316134 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1334/2660] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:20.845352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:21.707768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:27.957326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c778-eb3b-4666-a3b7-4615dba27a5c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:27.958517 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c778-eb3b-4666-a3b7-4615dba27a5c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:27.958629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c778-eb3b-4666-a3b7-4615dba27a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:27.975896 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d982c778-eb3b-4666-a3b7-4615dba27a5c None None] MFA Rules not processed for user `09e3e81754b345009e5d2b976ca321f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:27.977097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d982c778-eb3b-4666-a3b7-4615dba27a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:28.031889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1327/2661] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:28.214669 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:28.281227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b78d-8d52-41c2-8f52-ba959de45d8d service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:28.285843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:28.286739 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:28.287019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:28.287293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] PATH_INFO: `/v3/users/01e9b0fa02c04198b53e877fb5400f8c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:28.288484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e9b0fa02c04198b53e877fb5400f8c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:28.290462 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:28.545110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e9b0fa02c04198b53e877fb5400f8c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:28.545848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e9b0fa02c04198b53e877fb5400f8c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:28.562626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e9b0fa02c04198b53e877fb5400f8c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:28.563703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e9b0fa02c04198b53e877fb5400f8c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:28.576094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e9b0fa02c04198b53e877fb5400f8c'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:28.661391 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1335/2662] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:28 2022] DELETE /identity/v3/users/01e9b0fa02c04198b53e877fb5400f8c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:47:28.666152 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:28.666508 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:28.666756 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:28.687069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:28.692596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:28.761396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1328/2663] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:28.773264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:28.775707 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:28.845486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dc71f-6207-42c8-a096-7534fb7fb9db admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:28.849595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:28.850261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:28.850540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:28.850808 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:28.860020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:28.861430 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:28.871356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1336/2664] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:29.288338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:29.288338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:29.300551 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530efb1-9a6c-446d-b8a2-2878988d0078 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:29.305478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:29.306497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:29.306869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:29.307206 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] PATH_INFO: `/v3/projects/5ae91e428f12454192d5fbfbdf0536db` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:29.308608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae91e428f12454192d5fbfbdf0536db)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:29.311155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:29.439704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae91e428f12454192d5fbfbdf0536db', 'request_id': 'req-132592f4-dcb0-4930-9891-2802d70b4ef0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.446667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae91e428f12454192d5fbfbdf0536db', 'request_id': 'req-132592f4-dcb0-4930-9891-2802d70b4ef0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.452339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae91e428f12454192d5fbfbdf0536db', 'request_id': 'req-132592f4-dcb0-4930-9891-2802d70b4ef0'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.568658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] The token cache is being invalidate because project 5ae91e428f12454192d5fbfbdf0536db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:29.568886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.569477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.601432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1329/2665] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:29 2022] DELETE /identity/v3/projects/5ae91e428f12454192d5fbfbdf0536db => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:29.601619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:29.601619 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32d1a7-4707-45d3-a24d-e60207369408 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:29.603988 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:29.642512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:29.643370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:29.643652 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:29.643929 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] PATH_INFO: `/v3/users/8af478845fe04d88b1ac0e8b8360c56a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:29.645095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af478845fe04d88b1ac0e8b8360c56a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:29.647040 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:29.893486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af478845fe04d88b1ac0e8b8360c56a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.894251 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af478845fe04d88b1ac0e8b8360c56a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.909277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af478845fe04d88b1ac0e8b8360c56a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.909277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af478845fe04d88b1ac0e8b8360c56a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:29.914436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-995f0f54-16fd-4318-ad1e-f297ab9a8dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af478845fe04d88b1ac0e8b8360c56a'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.045166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1337/2666] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:29 2022] DELETE /identity/v3/users/8af478845fe04d88b1ac0e8b8360c56a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:30.510032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:30.511532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:30.525380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132592f4-dcb0-4930-9891-2802d70b4ef0 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:30.554734 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:30.589554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:30.590438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:30.590714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:30.590983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] PATH_INFO: `/v3/projects/a6db376828b644dbbac754b1bc873417` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:30.592338 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6db376828b644dbbac754b1bc873417)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:30.595720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:30.723814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6db376828b644dbbac754b1bc873417', 'request_id': 'req-9a9ee772-06c8-4665-a36d-24a7c281f8e2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.730793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6db376828b644dbbac754b1bc873417', 'request_id': 'req-9a9ee772-06c8-4665-a36d-24a7c281f8e2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.736788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6db376828b644dbbac754b1bc873417', 'request_id': 'req-9a9ee772-06c8-4665-a36d-24a7c281f8e2'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.825824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] The token cache is being invalidate because project a6db376828b644dbbac754b1bc873417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:30.826116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.826817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9a9ee772-06c8-4665-a36d-24a7c281f8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:30.828428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1330/2667] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:30 2022] DELETE /identity/v3/projects/a6db376828b644dbbac754b1bc873417 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:47:30.833408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:30.833909 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:30.834011 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:30.852532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:30.853752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:30.929831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1338/2668] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:30.937913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:30.938381 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:30.938761 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:30.967453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:30.968869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:30.978155 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1331/2669] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:30.983827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.007118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84da775-d484-400c-8f2d-8a0d86203438 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.012153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.012841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.013119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.013388 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.014107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:47:31.014445 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.016403 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.032878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1339/2670] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:47:31.035282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.046009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca492d-2954-47dd-ace6-2b5ebf8e53ca None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.051419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.052768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.052768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.052768 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.053889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.055924 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.177147 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1332/2671] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:31 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:31.178813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.189453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2922949-331b-4107-8e36-416a2407dbc2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.218047 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:31.249854 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.250546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.250822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.251096 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.251919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.253683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.448946 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1340/2672] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:31 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:47:31.453397 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.464978 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addac7-224d-4d62-8a84-5685f5c67739 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.470515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.471176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.471451 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.471926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.472647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.474335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.487186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1333/2673] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:31.492108 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.502130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395831e9-beac-40a5-8ce8-5123f49f5252 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.506870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.507597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.507944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.508221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.509035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.510765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25025-d3c1-4fc2-97b0-5886f589bc0d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.521412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1341/2674] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:47:31.525713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.535837 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9010-44a1-4950-bbf2-fe9143639893 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.541085 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.541979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.542289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.542570 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] PATH_INFO: `/v3/projects/5d0985a9f48a4fd6aab8fe36f7fa1d84/users/55c1c1f1c30f4d4580a8eefe915cb0f7/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.559233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0985a9f48a4fd6aab8fe36f7fa1d84, user_id=55c1c1f1c30f4d4580a8eefe915cb0f7, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.561944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.669455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1334/2675] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:31 2022] PUT /identity/v3/projects/5d0985a9f48a4fd6aab8fe36f7fa1d84/users/55c1c1f1c30f4d4580a8eefe915cb0f7/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:31.680512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.680800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.681132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.712763 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] MFA Rules not processed for user `55c1c1f1c30f4d4580a8eefe915cb0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:31.713981 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:31.785827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1342/2676] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:31.791002 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.852063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:31.856939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2b96b-c978-4819-9c50-6e4b1f6c7e64 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:31.861892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:31.862596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:31.862942 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:31.863256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:31.864040 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:31.866124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:31.907544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.909758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:31.963889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1335/2677] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:31 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:31.964960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:31.967319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24883c07-526f-4bb1-9b23-7468550a1b0c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.032866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.032866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.032866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.032866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.033245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:32.033245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:32.033994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ee237-d3e6-4c15-b163-425827811261 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.036025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.037945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.042469 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.043457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.043819 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.044195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.045039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.047167 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1343/2678] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:47:32.047313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.163626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1336/2679] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:31 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:32.165087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.177650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf98f7-28bf-4e3e-98bb-651330aedadb service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.182367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.183025 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.183306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.183604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.184781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.185930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.197292 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1344/2680] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:32.201364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.212454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7697a-f9eb-4d0a-a712-86f11217a9db admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.231038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.231038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.231038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.231038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.231456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.231456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894576ae-a949-40eb-8136-14ae7d335a6c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.235520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1337/2681] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:32.239895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.250811 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24f9a6-3e60-4f2e-82fd-9e29b780c9ca admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.256922 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.257982 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.258296 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.260248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] PATH_INFO: `/v3/projects/15cf4fec53ea4dae8c94627aca8d778f/users/1eed7ba4d336444ba0163961dbce17cd/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.275987 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cf4fec53ea4dae8c94627aca8d778f, user_id=1eed7ba4d336444ba0163961dbce17cd, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.278720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.394941 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1345/2682] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:32 2022] PUT /identity/v3/projects/15cf4fec53ea4dae8c94627aca8d778f/users/1eed7ba4d336444ba0163961dbce17cd/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:32.406979 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.407270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.407552 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.459122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] MFA Rules not processed for user `1eed7ba4d336444ba0163961dbce17cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:32.460999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:32.529973 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1338/2683] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:32.534452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.597875 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0e67d-23a4-4d3e-a959-14bdf07e3e86 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.602791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.603436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.603718 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.603994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.604678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.606604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.732438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1346/2684] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:32 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:32.736732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.748686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9add5-5da9-4afa-bee8-4eb6e21c4c0b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.795128 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:32.810950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.811637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.811921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.812195 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.812888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.814881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:32.969720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1339/2685] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:32 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:47:32.973719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:32.987166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26db9d-f4ba-4ba3-8bfb-74dee22833c1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:32.994159 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:32.997908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:32.997908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:32.997908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:32.997908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:32.998529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:33.011613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1347/2686] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:33.015960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:33.026999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47b7d9-1ecd-4fcb-8d8d-a1105f4c97fd admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:33.031796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:33.032833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:33.033115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:33.033423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] PATH_INFO: `/v3/projects/883685ec2f904b3ea9c644ed4ad39297/users/f32b01a9a7504884b8beebe945942e2b/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:33.050684 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=883685ec2f904b3ea9c644ed4ad39297, user_id=f32b01a9a7504884b8beebe945942e2b, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:33.053976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:33.158021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1340/2687] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:33 2022] PUT /identity/v3/projects/883685ec2f904b3ea9c644ed4ad39297/users/f32b01a9a7504884b8beebe945942e2b/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:33.161772 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:33.172780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff278c-4cf8-4a96-87fa-4dbe7b67154d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:33.232441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:33.233540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:33.233824 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:33.234093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:33.234841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:33.236435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e39ef2-362e-4a9f-ac54-4e1d94782981 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:33.247201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1348/2688] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:33.251575 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:33.262171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc171-fd35-42ef-9833-252ec637c03c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:33.268889 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:33.269892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:33.270102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:33.270384 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] PATH_INFO: `/v3/projects/883685ec2f904b3ea9c644ed4ad39297/users/f32b01a9a7504884b8beebe945942e2b/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:33.272577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] RBAC: Authorizing `identity:create_grant(project_id=883685ec2f904b3ea9c644ed4ad39297, user_id=f32b01a9a7504884b8beebe945942e2b, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:33.275296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:33.289929 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1341/2689] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:33 2022] PUT /identity/v3/projects/883685ec2f904b3ea9c644ed4ad39297/users/f32b01a9a7504884b8beebe945942e2b/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:47:33.300529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771850-f423-4399-b125-b78adf032f9c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:33.300819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771850-f423-4399-b125-b78adf032f9c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:33.301098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771850-f423-4399-b125-b78adf032f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:33.330588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-be771850-f423-4399-b125-b78adf032f9c None None] MFA Rules not processed for user `f32b01a9a7504884b8beebe945942e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:33.331903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-be771850-f423-4399-b125-b78adf032f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:33.353588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:33.406543 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1349/2690] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:47:33.420668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:33.422415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:33.485706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee801d-4fe2-46b6-b4a1-774e9fcbecef admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:33.489634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:33.490311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:33.490620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:33.490985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:33.552952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:33.554486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:33.563956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1342/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:36.717864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771850-f423-4399-b125-b78adf032f9c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:36.728263 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771850-f423-4399-b125-b78adf032f9c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:36.732667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:36.733386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:36.733774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:36.734118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:36.796019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:36.797769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:36.806976 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1350/2692] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:38.441398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:38.452029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cae2f-2802-4334-ab43-832782390bb6 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:38.455881 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:38.456651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:38.457034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:38.457409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:38.468145 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:38.469616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454a2b4-b537-4db1-b2dc-6f9d80e6bc77 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:38.481354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1343/2693] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:43.542603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:43.606631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14b204-89a6-4abe-bf1d-ddd93cdafe24 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:43.611299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:43.612168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:43.612456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:43.612734 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] PATH_INFO: `/v3/users/55c1c1f1c30f4d4580a8eefe915cb0f7` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:43.613967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c1c1f1c30f4d4580a8eefe915cb0f7)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:43.615937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:43.964084 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c1c1f1c30f4d4580a8eefe915cb0f7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:43.964731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c1c1f1c30f4d4580a8eefe915cb0f7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:43.971812 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:43.977313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c1c1f1c30f4d4580a8eefe915cb0f7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:43.977949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c1c1f1c30f4d4580a8eefe915cb0f7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:43.984670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c1c1f1c30f4d4580a8eefe915cb0f7'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:44.067892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1351/2694] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:43 2022] DELETE /identity/v3/users/55c1c1f1c30f4d4580a8eefe915cb0f7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:44.073415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:44.073952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:44.074084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:44.092056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:44.093194 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:44.133361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:44.156104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1344/2695] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:44.166714 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:44.168277 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:44.230680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b36675-1bd0-4aee-bc0b-8e3ae5d3dcc0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:44.234320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:44.234961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:44.235243 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:44.235522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:44.245699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:44.247188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:44.257071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1352/2696] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:44.769430 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:44.770909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:44.783463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fd22f-262a-44da-a851-2b17224bbce0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:44.788380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:44.792398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:44.792398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:44.792398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] PATH_INFO: `/v3/projects/5d0985a9f48a4fd6aab8fe36f7fa1d84` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:44.792715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0985a9f48a4fd6aab8fe36f7fa1d84)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:44.794800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:44.953577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0985a9f48a4fd6aab8fe36f7fa1d84', 'request_id': 'req-65c46ab0-8381-4917-8d14-478028cabf74'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:44.959731 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0985a9f48a4fd6aab8fe36f7fa1d84', 'request_id': 'req-65c46ab0-8381-4917-8d14-478028cabf74'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:44.965263 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0985a9f48a4fd6aab8fe36f7fa1d84', 'request_id': 'req-65c46ab0-8381-4917-8d14-478028cabf74'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.050663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] The token cache is being invalidate because project 5d0985a9f48a4fd6aab8fe36f7fa1d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:45.050868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.051458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.053011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1345/2697] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:44 2022] DELETE /identity/v3/projects/5d0985a9f48a4fd6aab8fe36f7fa1d84 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:45.057823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:45.067690 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d28b1-d215-4f58-86ec-426666bb4811 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:45.128066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:45.128918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:45.129216 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:45.129472 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] PATH_INFO: `/v3/users/1eed7ba4d336444ba0163961dbce17cd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:45.130679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eed7ba4d336444ba0163961dbce17cd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:45.132576 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:45.329716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eed7ba4d336444ba0163961dbce17cd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.330459 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eed7ba4d336444ba0163961dbce17cd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.343211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eed7ba4d336444ba0163961dbce17cd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.343883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eed7ba4d336444ba0163961dbce17cd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.351136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eed7ba4d336444ba0163961dbce17cd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:45.355415 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:45.442631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1353/2698] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:45 2022] DELETE /identity/v3/users/1eed7ba4d336444ba0163961dbce17cd => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:45.972676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:45.974465 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:45.991844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c46ab0-8381-4917-8d14-478028cabf74 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:46.057988 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:46.059061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:46.059252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:46.059523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] PATH_INFO: `/v3/projects/15cf4fec53ea4dae8c94627aca8d778f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:46.060739 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cf4fec53ea4dae8c94627aca8d778f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:46.062668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:46.194266 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cf4fec53ea4dae8c94627aca8d778f', 'request_id': 'req-ced23b51-7b5a-4210-a87e-2d478d78f412'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.203018 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cf4fec53ea4dae8c94627aca8d778f', 'request_id': 'req-ced23b51-7b5a-4210-a87e-2d478d78f412'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.208869 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cf4fec53ea4dae8c94627aca8d778f', 'request_id': 'req-ced23b51-7b5a-4210-a87e-2d478d78f412'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.212471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:46.290154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] The token cache is being invalidate because project 15cf4fec53ea4dae8c94627aca8d778f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:46.290504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.291156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.292741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1346/2699] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:45 2022] DELETE /identity/v3/projects/15cf4fec53ea4dae8c94627aca8d778f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:46.297249 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:46.299349 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:46.312114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740f8a7-9579-43c0-a7a4-af58cbdd0cab admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:46.370094 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:46.370990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:46.371271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:46.371544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] PATH_INFO: `/v3/users/f32b01a9a7504884b8beebe945942e2b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:46.372750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32b01a9a7504884b8beebe945942e2b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:46.374716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:46.656257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32b01a9a7504884b8beebe945942e2b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.657017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32b01a9a7504884b8beebe945942e2b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.670463 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32b01a9a7504884b8beebe945942e2b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.671738 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32b01a9a7504884b8beebe945942e2b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.679117 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-8456b4b7-68ec-44ad-985b-86de379fd751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32b01a9a7504884b8beebe945942e2b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:46.786573 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1354/2700] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:47:46 2022] DELETE /identity/v3/users/f32b01a9a7504884b8beebe945942e2b => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:47.209652 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:47.211785 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:47.227262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced23b51-7b5a-4210-a87e-2d478d78f412 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:47.290260 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:47.291032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.291312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.291582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] PATH_INFO: `/v3/projects/883685ec2f904b3ea9c644ed4ad39297` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.292834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883685ec2f904b3ea9c644ed4ad39297)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:47.294832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:47.407450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883685ec2f904b3ea9c644ed4ad39297', 'request_id': 'req-007361ab-19c5-4650-94a6-478e40b2c8b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:47.415121 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883685ec2f904b3ea9c644ed4ad39297', 'request_id': 'req-007361ab-19c5-4650-94a6-478e40b2c8b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:47.420895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883685ec2f904b3ea9c644ed4ad39297', 'request_id': 'req-007361ab-19c5-4650-94a6-478e40b2c8b4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:47.519985 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] The token cache is being invalidate because project 883685ec2f904b3ea9c644ed4ad39297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:47:47.520363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:47.521290 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-007361ab-19c5-4650-94a6-478e40b2c8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:47:47.523057 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1347/2701] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:47:47 2022] DELETE /identity/v3/projects/883685ec2f904b3ea9c644ed4ad39297 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:47.530097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.530601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.530788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.550777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:47.551920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:47.619397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1355/2702] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:47.628407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.628691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.628970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.654287 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:47.655549 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:47.664741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1348/2703] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:47:47.669452 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:47.680841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d8c29-318b-4b31-bb93-de4f903dca5d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:47.686102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:47.686962 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.687246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.687524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.688309 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:47:47.688673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:47.690396 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:47.706069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1356/2704] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:47:47.710698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:47.720937 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc05a-58ad-411a-b8fc-7ab9995a3069 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:47.725866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:47.727156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.727434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.727712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.728412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:47.730411 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:47.830175 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1349/2705] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:47:47 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:47:47.834770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:47.840482 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:47.846731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fe7fe-d3de-482c-a18c-5a039a689159 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:47.910183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:47.910870 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:47.911153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:47.911432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:47.912130 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:47.914081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:47.988299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1357/2706] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:47:47 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:47:47.992523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:48.003282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d168a9-0fb7-42f0-8182-7f1af4eeda9c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:48.009132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:48.009797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:48.010160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:48.010377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:48.011115 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:48.012722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:48.025982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1350/2707] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:48.030840 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:48.041215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e35b72-515e-459d-8067-34f5876594de admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:48.045885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:48.046816 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:48.047101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:48.047426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:48.048085 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:48.049758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878702c-97a6-49e6-ba0d-5b80aca61a36 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:48.060498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1358/2708] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:47:48.064672 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:48.075393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6ee3-585a-4f5e-b1c7-54b00ab9dd70 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:48.080433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:48.081261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:48.081543 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:48.081822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] PATH_INFO: `/v3/projects/cda873aeae3e4e6480ebe0ff7074f4bc/users/fb92393ab99b4b938d05e8aec6b7627f/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:48.098948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda873aeae3e4e6480ebe0ff7074f4bc, user_id=fb92393ab99b4b938d05e8aec6b7627f, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:48.102323 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:48.188463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1351/2709] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:47:48 2022] PUT /identity/v3/projects/cda873aeae3e4e6480ebe0ff7074f4bc/users/fb92393ab99b4b938d05e8aec6b7627f/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:47:48.199799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:48.205106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:48.205106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:48.233434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] MFA Rules not processed for user `fb92393ab99b4b938d05e8aec6b7627f`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:47:48.235389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:48.307039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1359/2710] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:47:48.318493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:47:48.321089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:47:48.350745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:48.381271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db21ef-13d2-42b6-b604-87faddc471b7 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:47:48.385550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:47:48.386215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:47:48.386507 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:47:48.386905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:47:48.396200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:47:48.397743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:47:48.407717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1352/2711] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:47:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:47:58.303293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:47:58.500364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=16 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:48:16.599839 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:16.601257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:16.738802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbedc9-7ae5-4090-8d15-a123af0e92b1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:16.745008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:16.745641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:16.745919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:16.746209 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:16.753045 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:16.760145 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:16.761751 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64399a24-946c-4e2f-a96c-52f99dedfffb service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:16.772314 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1360/2712] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:48:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:48:26.771019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=56 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:48:53.603139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:53.604490 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:53.742715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f65e8-94fe-408c-84a1-3b9f862ca0b3 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:53.749101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:53.750026 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:53.750341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:53.750616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] PATH_INFO: `/v3/users/09e3e81754b345009e5d2b976ca321f4` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:53.751825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e3e81754b345009e5d2b976ca321f4)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:53.753765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:53.959608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:53.960194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:53.960194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:53.979093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] MFA Rules not processed for user `6772c708b48b404bb9f8794142ec6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:48:53.980181 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:54.034125 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e3e81754b345009e5d2b976ca321f4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.039421 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1361/2713] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:48:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:48:54.041743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e3e81754b345009e5d2b976ca321f4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.055058 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e3e81754b345009e5d2b976ca321f4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.055645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e3e81754b345009e5d2b976ca321f4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.224391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e3e81754b345009e5d2b976ca321f4'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.231200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1353/2714] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:48:53 2022] DELETE /identity/v3/users/09e3e81754b345009e5d2b976ca321f4 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:48:54.617037 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:54.628841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75a14f-f4a2-4f9b-88ac-e29171fc0a65 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:54.704116 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:54.705028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:54.705351 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:54.705668 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] PATH_INFO: `/v3/projects/1c6cf7096e63465d8e861f9f82b21e2b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:54.707019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c6cf7096e63465d8e861f9f82b21e2b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:54.709223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:54.807544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:54.809047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:54.855667 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:48:54.898996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c6cf7096e63465d8e861f9f82b21e2b', 'request_id': 'req-45382e4c-efc2-4a14-a2e4-653b2eb87a22'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.908033 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c6cf7096e63465d8e861f9f82b21e2b', 'request_id': 'req-45382e4c-efc2-4a14-a2e4-653b2eb87a22'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.908200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450ab4d-6886-4d57-b842-3dc522abd104 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:54.913716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c6cf7096e63465d8e861f9f82b21e2b', 'request_id': 'req-45382e4c-efc2-4a14-a2e4-653b2eb87a22'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:54.970716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:54.971364 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:54.971644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:54.971921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:54.973715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:54.986326 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:54.987950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:55.000395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1354/2715] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:48:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:48:55.082453 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] The token cache is being invalidate because project 1c6cf7096e63465d8e861f9f82b21e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:48:55.082991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:55.083927 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-45382e4c-efc2-4a14-a2e4-653b2eb87a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:55.090301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1362/2716] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:48:54 2022] DELETE /identity/v3/projects/1c6cf7096e63465d8e861f9f82b21e2b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:48:59.208170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:59.209745 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:59.291270 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f3ddc-4f79-4d4e-96b4-620c79add2be service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:59.296708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:59.297583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:59.297860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:59.298132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] PATH_INFO: `/v3/users/fb92393ab99b4b938d05e8aec6b7627f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:59.299405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb92393ab99b4b938d05e8aec6b7627f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:59.301335 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:59.536998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb92393ab99b4b938d05e8aec6b7627f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:59.547595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb92393ab99b4b938d05e8aec6b7627f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:59.557583 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:48:59.562600 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb92393ab99b4b938d05e8aec6b7627f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:59.563596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb92393ab99b4b938d05e8aec6b7627f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:59.688581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb92393ab99b4b938d05e8aec6b7627f'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:48:59.696839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1355/2717] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:48:59 2022] DELETE /identity/v3/users/fb92393ab99b4b938d05e8aec6b7627f => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:48:59.702109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:59.702761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:59.702761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:59.734972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:48:59.736215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:59.761750 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:48:59.800184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1363/2718] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:48:59.812741 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:48:59.814486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:48:59.919955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484311-6184-449f-9d40-330d47aa7c45 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:48:59.924638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:48:59.925357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:48:59.925644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:48:59.925923 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:48:59.936744 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:48:59.938412 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfc689-814d-4480-9abe-dcd139362fcd service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:48:59.949625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1356/2719] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:00.540432 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:00.541804 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:00.555440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79b556-63b7-43c3-adfa-6a42b0349363 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:00.562433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:00.563315 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:00.563768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:00.563889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] PATH_INFO: `/v3/projects/cda873aeae3e4e6480ebe0ff7074f4bc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:00.565386 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda873aeae3e4e6480ebe0ff7074f4bc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:00.568320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:00.730166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda873aeae3e4e6480ebe0ff7074f4bc', 'request_id': 'req-a1526ee3-707c-4c78-8d2d-1f0104790da0'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:00.737009 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda873aeae3e4e6480ebe0ff7074f4bc', 'request_id': 'req-a1526ee3-707c-4c78-8d2d-1f0104790da0'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:00.742643 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda873aeae3e4e6480ebe0ff7074f4bc', 'request_id': 'req-a1526ee3-707c-4c78-8d2d-1f0104790da0'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:00.854708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] The token cache is being invalidate because project cda873aeae3e4e6480ebe0ff7074f4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:49:00.854938 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:00.855656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-a1526ee3-707c-4c78-8d2d-1f0104790da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:00.857728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1364/2720] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:49:00 2022] DELETE /identity/v3/projects/cda873aeae3e4e6480ebe0ff7074f4bc => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:00.863061 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:00.867650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:00.867650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:00.918487 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:00.919720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:00.980097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1357/2721] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:00.988720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:00.989006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:00.989284 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.008720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:01.009936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:01.019078 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1365/2722] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:01.025790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.035843 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc026a88-01f2-4ef8-a5a3-338ab1a12cd6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.040615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.041277 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.041553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.041822 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.042581 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:49:01.042885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.044628 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.058069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1358/2723] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:49:01.063013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.072680 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f56bd4-4a92-41f1-b101-53864c15d7f8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.077503 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.078161 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.078481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.078753 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.079460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.081990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.186335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1366/2724] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:49:01 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:01.190405 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.201354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31732a6f-798a-43d6-acfa-8bdf8e2bc95a admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.229463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:01.266255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.266976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.267262 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.267540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.268313 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.270934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.425657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1359/2725] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:49:01 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:01.429836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.441065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf6da-4c91-4f0b-8cef-e8408611816b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.445640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.446325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.446614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.446894 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.447642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.449242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.460178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1367/2726] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:01.464714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.474847 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c096e9-4803-4525-9a7d-ce86a83fbfe8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.479501 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.480154 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.480433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.480706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.481513 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.483718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c59dc-4152-41f0-b9eb-af78916a781a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.493873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1360/2727] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:01.498608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.508473 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10c9aa-3869-4de4-98ed-74b732f6e80e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.513028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.513896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.514176 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.514518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] PATH_INFO: `/v3/projects/2ce16e6ee5e744d882a10a967fe64ad2/users/a7ec3ba35ee94014aba246c60269bfe1/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.531019 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce16e6ee5e744d882a10a967fe64ad2, user_id=a7ec3ba35ee94014aba246c60269bfe1, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.533830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.630142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1368/2728] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:49:01 2022] PUT /identity/v3/projects/2ce16e6ee5e744d882a10a967fe64ad2/users/a7ec3ba35ee94014aba246c60269bfe1/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:01.642337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81ad5-b515-4dd9-ae90-4cb46aff44ca None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.642627 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81ad5-b515-4dd9-ae90-4cb46aff44ca None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.642905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd81ad5-b515-4dd9-ae90-4cb46aff44ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.672336 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-edd81ad5-b515-4dd9-ae90-4cb46aff44ca None None] MFA Rules not processed for user `a7ec3ba35ee94014aba246c60269bfe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:01.673943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-edd81ad5-b515-4dd9-ae90-4cb46aff44ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:01.739990 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1361/2729] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:01.756394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:01.758993 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:01.821873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:01.846485 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc782fd-f96c-4e35-8655-71dff4d5d7a9 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:01.850426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:01.851061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:01.851384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:01.851657 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:01.861422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:01.862890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:01.871960 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1369/2730] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:03.744392 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211158a9-3dfa-4e32-b300-aa985671f4af None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:03.745067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211158a9-3dfa-4e32-b300-aa985671f4af None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:03.745067 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211158a9-3dfa-4e32-b300-aa985671f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:03.762860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-211158a9-3dfa-4e32-b300-aa985671f4af None None] MFA Rules not processed for user `a7ec3ba35ee94014aba246c60269bfe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:03.764056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-211158a9-3dfa-4e32-b300-aa985671f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:03.770931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1362/2731] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:07.187559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:07.188691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:07.295197 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384034f-22e4-42ff-ba8c-949ea26b6cb3 service cinder] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:07.306558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:07.307570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:07.308104 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:07.308611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:07.310788 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:07.335282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:07.336807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:07.350273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1370/2732] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:13.757668 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:17.345731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=47 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:25.031423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5c2c-f751-420d-ad29-0993a61e70a5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:25.031943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5c2c-f751-420d-ad29-0993a61e70a5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:25.031943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5c2c-f751-420d-ad29-0993a61e70a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:25.050634 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ec0f5c2c-f751-420d-ad29-0993a61e70a5 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:25.051821 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ec0f5c2c-f751-420d-ad29-0993a61e70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:25.105925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1363/2733] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:25.267477 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:25.268908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:25.336807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b04b9-1dfb-45fc-848c-c0f77677d94b service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:25.342465 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:25.343111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:25.343393 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:25.343665 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:25.351768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:25.353334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:25.361206 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1371/2734] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:49:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:49:26.285963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a90f0-1de6-4770-b1f4-c728b05491b9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:26.286493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a90f0-1de6-4770-b1f4-c728b05491b9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:26.286608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a90f0-1de6-4770-b1f4-c728b05491b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:26.304630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-047a90f0-1de6-4770-b1f4-c728b05491b9 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:26.305820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-047a90f0-1de6-4770-b1f4-c728b05491b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:26.312939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1364/2735] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:49:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:26.325558 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:26.341407 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6fe15-574a-4d89-8c4d-a2e6d9bd5877 service swift] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:26.341890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:26.342563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:26.342836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:26.343107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:26.351605 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:26.353170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:26.361202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1372/2736] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:49:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 09:49:27.078993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151fa43-8c17-4e55-8c4a-2252bbc98765 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:27.079285 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151fa43-8c17-4e55-8c4a-2252bbc98765 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:27.079565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151fa43-8c17-4e55-8c4a-2252bbc98765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:27.101934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4151fa43-8c17-4e55-8c4a-2252bbc98765 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:27.101934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4151fa43-8c17-4e55-8c4a-2252bbc98765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:27.109161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1365/2737] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:49:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:27.122035 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:27.133238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207fbc2-d519-4950-a21e-43fa6875ff6e service swift] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:27.138521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:27.139275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:27.139556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:27.139828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:27.148111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:27.149777 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:27.158621 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1373/2738] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:49:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:49:27.763485 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:27.763945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:27.764065 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:27.781910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:27.783321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:27.790396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1366/2739] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:49:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:27.803109 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:27.814537 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c8115-f516-4b36-8640-01593b464d30 service swift] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:27.819683 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:27.820308 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:27.820590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:27.820863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:27.829723 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:27.831331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91bfe8-870d-4217-9ae6-b518b3a7fd9b service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:27.839792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1374/2740] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:49:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:49:30.834830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:30.897148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb8a2-d223-42c6-97de-0f27a683e99d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:30.901644 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:30.902538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:30.902816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:30.903651 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] PATH_INFO: `/v3/users/a7ec3ba35ee94014aba246c60269bfe1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:30.904853 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ec3ba35ee94014aba246c60269bfe1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:30.906801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:31.171527 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ec3ba35ee94014aba246c60269bfe1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.178453 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ec3ba35ee94014aba246c60269bfe1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.190177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ec3ba35ee94014aba246c60269bfe1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.190939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ec3ba35ee94014aba246c60269bfe1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.311124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ec3ba35ee94014aba246c60269bfe1'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.317489 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1367/2741] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:49:30 2022] DELETE /identity/v3/users/a7ec3ba35ee94014aba246c60269bfe1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:31.322337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:31.322861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:31.322861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:31.342765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:31.343966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:31.401710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1375/2742] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:31.411874 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:31.416909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:31.419781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:31.470721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62acd9-e0c4-4d4a-ae60-dd61834e7789 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:31.474157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:31.474825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:31.475098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:31.475408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:31.484429 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:31.485848 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4466df6-4213-405d-b152-feaa46d2b5b6 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:31.495169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1368/2743] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:31.752916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:31.757056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:31.766306 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881db657-4c90-4ee9-ba06-0dff44e7945e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:31.770958 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:31.771810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:31.772086 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:31.772360 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] PATH_INFO: `/v3/projects/2ce16e6ee5e744d882a10a967fe64ad2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:31.773539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce16e6ee5e744d882a10a967fe64ad2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:31.775464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:31.884201 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:31.887781 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce16e6ee5e744d882a10a967fe64ad2', 'request_id': 'req-85796e9b-4e68-4612-83dc-679e0ab67016'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.894023 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce16e6ee5e744d882a10a967fe64ad2', 'request_id': 'req-85796e9b-4e68-4612-83dc-679e0ab67016'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:31.899583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce16e6ee5e744d882a10a967fe64ad2', 'request_id': 'req-85796e9b-4e68-4612-83dc-679e0ab67016'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:32.004907 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] The token cache is being invalidate because project 2ce16e6ee5e744d882a10a967fe64ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:49:32.005211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:32.005864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-85796e9b-4e68-4612-83dc-679e0ab67016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:49:32.007538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1376/2744] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:49:31 2022] DELETE /identity/v3/projects/2ce16e6ee5e744d882a10a967fe64ad2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:49:32.012968 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.015944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.015944 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.032692 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:32.034031 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:32.093104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1369/2745] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:32.100933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.101214 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.101487 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.119538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:32.120721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:32.128761 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1377/2746] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:32.132971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.143906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e33357-ecf9-4f15-a2f1-38c4e44fce14 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.148449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.149087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.149369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.149641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.150399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:49:32.150703 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.152286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.164224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1370/2747] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:49:32.168419 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.180148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527f9da-2ac9-477c-bc49-4c1722391b01 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.184893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.185544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.185829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.186103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.186823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.188716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.331428 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1378/2748] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:49:32 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:32.335586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.346909 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f461b-7083-47bc-bb72-45b8cece79d8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.407051 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.407805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.408086 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.408360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.409069 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.411030 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.507164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1371/2749] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:49:32 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:32.510148 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.521362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac465e69-e87d-4beb-83cb-c95cbc8c0b93 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.526271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.527008 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.528053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.528053 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.528276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.529892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.541154 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1379/2750] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:32.545481 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.555573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704e457-160f-4dad-a6d3-4c0c9062c518 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.560735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.561395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.561676 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.561953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.562708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.564324 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4b0f-0cc2-453d-9c26-128e4b6a5adf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.576354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1372/2751] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:32.580642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.591066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57dc17-1067-49f0-9978-076b926403fa admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.595702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.596568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.596848 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.597120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] PATH_INFO: `/v3/projects/c8c6682d39b441fd9d58defd883f3e9a/users/25b3c1ea6b294c12a678d746cd46c72d/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.612437 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c6682d39b441fd9d58defd883f3e9a, user_id=25b3c1ea6b294c12a678d746cd46c72d, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.615146 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.717895 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1380/2752] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:49:32 2022] PUT /identity/v3/projects/c8c6682d39b441fd9d58defd883f3e9a/users/25b3c1ea6b294c12a678d746cd46c72d/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:32.723101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf53-5d04-4988-8027-27af8a9d1917 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.723388 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf53-5d04-4988-8027-27af8a9d1917 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.723665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf53-5d04-4988-8027-27af8a9d1917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.742710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6a3cdf53-5d04-4988-8027-27af8a9d1917 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:32.743916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6a3cdf53-5d04-4988-8027-27af8a9d1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:32.753027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:32.804076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1373/2753] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:49:32.814831 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:32.816570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:32.874106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964274d8-c496-42c6-8eac-6ad9a9d2bf89 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:32.877624 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:32.878286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:32.878569 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:32.878843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:32.887883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:32.889294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:32.898538 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1381/2754] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:39.289956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.290591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.290591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.320139 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] MFA Rules not processed for user `25b3c1ea6b294c12a678d746cd46c72d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:39.321396 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:39.403608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1374/2755] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:39.408013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.417837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb20f-48e0-4f1c-8736-6c8ecd1d424a service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.423438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.424267 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.424559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.424836 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.425554 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:39.427612 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:39.510693 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1382/2756] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:49:39 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:39.515272 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.526618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb16a2-1719-4a7a-971e-a814d686c38f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.591964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.592639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.592951 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.593223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.593960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:39.595940 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:39.717911 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1375/2757] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:49:39 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:49:39.722814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.732491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493b770-9d44-462a-a68f-9404c0a69de4 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.744727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.744727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.744727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.744727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.747164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:39.747164 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:39.753782 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1383/2758] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:39.758895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.769794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6dee-86ff-450c-ba90-4dcec667efc9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.775003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.776805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.776805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.776805 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] PATH_INFO: `/v3/projects/37a180406f5b47889404485f32614e61/users/b39bda2ffefb4025aa36e8ef7b0103d9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.795767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a180406f5b47889404485f32614e61, user_id=b39bda2ffefb4025aa36e8ef7b0103d9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:39.798824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:39.865594 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1376/2759] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:49:39 2022] PUT /identity/v3/projects/37a180406f5b47889404485f32614e61/users/b39bda2ffefb4025aa36e8ef7b0103d9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:39.868147 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.879600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4d23e-01fa-4c83-8cb3-7f8445657da5 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.927056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:39.952821 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.954028 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.954608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.955118 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:39.956505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:39.959434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:39.970971 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1384/2760] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:49:39.975160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:39.991448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f446-46ac-4b7c-ad1a-8b1bb4113967 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:39.996205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:39.997637 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:39.997913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:39.998187 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] PATH_INFO: `/v3/projects/37a180406f5b47889404485f32614e61/users/b39bda2ffefb4025aa36e8ef7b0103d9/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:40.000261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a180406f5b47889404485f32614e61, user_id=b39bda2ffefb4025aa36e8ef7b0103d9, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:40.002980 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d300816-ab35-499c-9f41-258222f0f018 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:40.006733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:40.016810 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1377/2761] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:49:39 2022] PUT /identity/v3/projects/37a180406f5b47889404485f32614e61/users/b39bda2ffefb4025aa36e8ef7b0103d9/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:49:42.320484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:42.323338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:42.469226 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f963ee-4827-41bd-bf42-d90dae0638f2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:42.596645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:42.596645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:42.596645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:42.596645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:42.599956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:42.614372 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:42.616067 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:42.629878 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1385/2762] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:49:45.948377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335256-a110-436a-b555-e6d829e70468 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:45.948949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335256-a110-436a-b555-e6d829e70468 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:45.948949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335256-a110-436a-b555-e6d829e70468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:45.983236 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-12335256-a110-436a-b555-e6d829e70468 None None] MFA Rules not processed for user `b39bda2ffefb4025aa36e8ef7b0103d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:45.984478 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-12335256-a110-436a-b555-e6d829e70468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:46.094796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1378/2763] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:49:46.104638 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:46.106522 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:46.168128 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:46.196375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901232-5178-46fd-8e4e-af5f3e99e533 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:46.201534 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8e09e92e284747b80ca816b6e15a2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:46.202220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:46.202523 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:46.202961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:46.267491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:46.268944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:46.278802 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1386/2764] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:49:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:49:46.539473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457cd0c-5f0c-4e76-b131-a9b90110b5c0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:46.539473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457cd0c-5f0c-4e76-b131-a9b90110b5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:46.539473 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457cd0c-5f0c-4e76-b131-a9b90110b5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:46.561796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7457cd0c-5f0c-4e76-b131-a9b90110b5c0 None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:46.563189 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7457cd0c-5f0c-4e76-b131-a9b90110b5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:46.633295 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1379/2765] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:49:46.649319 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:49:46.651992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:46.749920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040b49-7986-46a6-91b6-b60e36233c05 service glance] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:49:46.755071 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:49:46.755729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:46.756003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:46.756273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:46.768770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:49:46.771711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d4ad-809a-42bd-b558-2dd91a582118 service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:49:46.783762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1387/2766] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:49:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:49:56.627656 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=61 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:56.780122 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=67 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:49:57.918587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:49:57.919232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:49:57.919232 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:49:57.939276 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] MFA Rules not processed for user `25b3c1ea6b294c12a678d746cd46c72d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:49:57.940790 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:49:57.948537 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1380/2767] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:49:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:50:02.266252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:50:02.267113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:50:02.267113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:50:02.295074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:50:02.296863 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:50:02.313766 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1388/2768] 10.222.0.48 () {58 vars in 1009 bytes} [Fri May 27 09:50:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:50:02.326565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:50:02.339164 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff13d-2625-4dc6-af13-3d03e5f3b33c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:50:02.346114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:50:02.346945 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:50:02.347257 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:50:02.347657 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:50:02.360446 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:50:02.362534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d3ead-a382-46e5-9dbb-7596943a3d3f service swift] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:50:02.372315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1381/2769] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:50:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:50:12.286136 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:50:12.368486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=9 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:50:33.103883 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:50:33.105330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:50:33.162943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62804dba-5a85-4203-8b62-a737fde7465d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:50:33.166547 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:50:33.167189 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:50:33.167470 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:50:33.167746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:50:33.177141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:50:33.178609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:50:33.187966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1389/2770] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:50:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:50:43.183713 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=26 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:50:55.598721 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:50:55.599341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:50:55.599341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:50:55.619577 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] MFA Rules not processed for user `6772c708b48b404bb9f8794142ec6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:50:55.620917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:50:55.682024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1382/2771] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:50:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:50:55.995817 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:50:56.005305 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd67b1-4ba4-4314-94f1-fcf518e6297f service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:50:56.008975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:50:56.009601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:50:56.009884 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:50:56.010158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:50:56.011956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:50:56.024672 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:50:56.026083 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:50:56.034494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1390/2772] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:50:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:51:05.674365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=25 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:51:06.030841 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:51:40.650255 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:51:40.660599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057d48-3ebb-4427-a40d-ca58efaad304 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:51:40.664279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:51:40.664921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:51:40.665204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:51:40.665480 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:51:40.667852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:51:40.760648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:51:40.762151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:51:40.782690 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1383/2773] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:51:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:51:44.328135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:51:44.336872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c442dd7-eeb5-480d-b907-b7728c0b8a25 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:51:44.341348 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:51:44.341995 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:51:44.342352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:51:44.342644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:51:44.344438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:51:44.359230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:51:44.360198 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:51:44.371920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1391/2774] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:51:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:51:50.779926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=46 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:51:54.366721 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:52:26.760675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:52:26.762108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:52:26.776616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6137d8-1409-4804-9613-10271983dcee service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:52:26.782220 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:52:26.782886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:52:26.783168 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:52:26.783443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:52:26.785427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:52:26.797386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:52:26.799087 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:52:26.808322 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1384/2775] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:52:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:52:36.804103 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:52:38.238966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:52:38.247745 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ffb-dd77-48bd-bc6b-d1f18823ede5 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:52:38.251450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:52:38.252081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:52:38.252358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:52:38.252631 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:52:38.254501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:52:38.264475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:52:38.265921 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e1aa3-9e18-4074-b4ca-f46bb2daa2b2 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:52:38.274790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1392/2776] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:52:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:52:41.620292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:52:41.632216 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24021cfc-50bc-4114-ba1e-b84d6474ffa2 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:52:41.642349 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:52:41.643027 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:52:41.643306 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:52:41.643604 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:52:41.645503 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:52:41.659926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:52:41.661520 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881ff95-8547-4e78-b1e5-c36bf94ae377 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:52:41.673984 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1385/2777] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:52:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:52:46.907060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:52:46.907580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:52:46.907681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:52:46.925433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] MFA Rules not processed for user `6772c708b48b404bb9f8794142ec6060`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:52:46.926662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:52:46.933608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1393/2778] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:52:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:52:51.668781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:52:56.920572 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=9 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:53:10.644376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:10.644890 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:10.645004 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:10.702975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] MFA Rules not processed for user `86d945bffa964372bec61684a4a1e95a`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:53:10.704120 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:53:10.721243 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1386/2779] 10.222.0.48 () {58 vars in 967 bytes} [Fri May 27 09:53:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:53:10.762151 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:10.784339 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e148f0-2c09-47c4-b95a-b508aea41ec9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:10.789549 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '444436f77ef74d9fb825aafa19ccc146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:10.790221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:10.790511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:10.790791 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:10.799813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:10.801474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:10.810517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1394/2780] 10.222.0.48 () {60 vars in 1384 bytes} [Fri May 27 09:53:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 09:53:15.392877 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:15.405160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0032af-6a70-45d9-a08d-43634ea0395f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:15.410779 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:15.411553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:15.411840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:15.413893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:15.413893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:53:15.427282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:15.428885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:15.439802 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1387/2781] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:53:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:53:19.099691 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:19.111961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b407d-328e-40a7-bec1-54a7aa3acbb9 service swift] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:19.117375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:19.117985 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:19.118299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:19.118567 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:19.120368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:53:19.133291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:19.134918 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:19.145126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1395/2782] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:53:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:53:19.226142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:19.293943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe6be-6ccf-43e2-95d2-116414273443 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:19.298585 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:19.299450 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:19.299729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:19.300007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] PATH_INFO: `/v3/users/25b3c1ea6b294c12a678d746cd46c72d` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:19.301179 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b3c1ea6b294c12a678d746cd46c72d)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:19.303153 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:19.524546 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b3c1ea6b294c12a678d746cd46c72d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:19.531910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b3c1ea6b294c12a678d746cd46c72d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:19.545148 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b3c1ea6b294c12a678d746cd46c72d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:19.545913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b3c1ea6b294c12a678d746cd46c72d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:19.615625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b3c1ea6b294c12a678d746cd46c72d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:19.621934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1388/2783] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:53:19 2022] DELETE /identity/v3/users/25b3c1ea6b294c12a678d746cd46c72d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:53:19.851254 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:19.862501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080c10-28fb-4a5b-90ef-7b6b95e00546 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:19.917186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:19.918325 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:19.918602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:19.918872 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] PATH_INFO: `/v3/projects/c8c6682d39b441fd9d58defd883f3e9a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:19.920602 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c6682d39b441fd9d58defd883f3e9a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:19.922516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:20.025671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c6682d39b441fd9d58defd883f3e9a', 'request_id': 'req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.032329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c6682d39b441fd9d58defd883f3e9a', 'request_id': 'req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.037823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c6682d39b441fd9d58defd883f3e9a', 'request_id': 'req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.121580 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] The token cache is being invalidate because project c8c6682d39b441fd9d58defd883f3e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:53:20.122294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.122414 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.123997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1396/2784] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:53:19 2022] DELETE /identity/v3/projects/c8c6682d39b441fd9d58defd883f3e9a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:53:20.127971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:20.129366 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:53:20.142235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50023f-b39c-4714-8546-0ba8d61edf31 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:20.154987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:53:20.200165 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:20.201110 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:20.201387 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:20.201655 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] PATH_INFO: `/v3/users/b39bda2ffefb4025aa36e8ef7b0103d9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:20.202835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39bda2ffefb4025aa36e8ef7b0103d9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:20.204841 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:20.404778 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39bda2ffefb4025aa36e8ef7b0103d9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.411421 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39bda2ffefb4025aa36e8ef7b0103d9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.423383 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39bda2ffefb4025aa36e8ef7b0103d9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.424071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39bda2ffefb4025aa36e8ef7b0103d9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.624987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39bda2ffefb4025aa36e8ef7b0103d9'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:20.631491 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1389/2785] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:53:20 2022] DELETE /identity/v3/users/b39bda2ffefb4025aa36e8ef7b0103d9 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:53:20.895843 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:53:20.897272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:53:20.910153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ccd96-8b95-4bf6-a64b-0a34c2e66857 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:53:20.961852 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:53:20.968513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:53:20.969361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:53:20.969642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:53:20.969937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] PATH_INFO: `/v3/projects/37a180406f5b47889404485f32614e61` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:53:20.971162 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a180406f5b47889404485f32614e61)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:53:20.973103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:53:21.108127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a180406f5b47889404485f32614e61', 'request_id': 'req-6c06024d-85ac-42be-a812-237205e84e80'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:21.114495 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a180406f5b47889404485f32614e61', 'request_id': 'req-6c06024d-85ac-42be-a812-237205e84e80'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:21.119924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a180406f5b47889404485f32614e61', 'request_id': 'req-6c06024d-85ac-42be-a812-237205e84e80'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:21.255517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] The token cache is being invalidate because project 37a180406f5b47889404485f32614e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:53:21.256257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:21.256464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:53:21.258229 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1397/2786] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:53:20 2022] DELETE /identity/v3/projects/37a180406f5b47889404485f32614e61 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:53:30.627795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:53:31.251119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:54:40.614056 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:40.615590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:40.673011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7a791-9ee9-4a39-a966-1a54e2e223b1 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:40.721931 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:40.722630 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:40.722910 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:40.723180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:40.724969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:40.787230 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:40.788615 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:40.798400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1390/2787] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:54:44.419529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:44.421000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:44.433692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06024d-85ac-42be-a812-237205e84e80 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:44.440611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:44.441509 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:44.441774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:44.442051 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] PATH_INFO: `/v3/users/6772c708b48b404bb9f8794142ec6060` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:44.443279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6772c708b48b404bb9f8794142ec6060)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:44.445257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:44.645367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6772c708b48b404bb9f8794142ec6060'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:44.646010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6772c708b48b404bb9f8794142ec6060'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:44.658330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6772c708b48b404bb9f8794142ec6060'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:44.659013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6772c708b48b404bb9f8794142ec6060'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:44.666003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-7b60df35-3449-404b-9aba-373a69fd8687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6772c708b48b404bb9f8794142ec6060'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:44.736275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1398/2788] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:54:44 2022] DELETE /identity/v3/users/6772c708b48b404bb9f8794142ec6060 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:54:44.981395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:44.993158 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a073ab-91e0-44f9-af03-65719c6442a3 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.051885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.052732 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.053011 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.053284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] PATH_INFO: `/v3/projects/1b2ae84b95ae44d18119450b94735f03` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.054488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2ae84b95ae44d18119450b94735f03)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.056439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.064872 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:54:45.168700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2ae84b95ae44d18119450b94735f03', 'request_id': 'req-03f33ebb-d6fe-4448-8740-6c852654879c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:45.175078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2ae84b95ae44d18119450b94735f03', 'request_id': 'req-03f33ebb-d6fe-4448-8740-6c852654879c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:45.180823 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2ae84b95ae44d18119450b94735f03', 'request_id': 'req-03f33ebb-d6fe-4448-8740-6c852654879c'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:45.275545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] The token cache is being invalidate because project 1b2ae84b95ae44d18119450b94735f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:54:45.275955 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:45.276726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-03f33ebb-d6fe-4448-8740-6c852654879c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:54:45.278597 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1391/2789] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:54:44 2022] DELETE /identity/v3/projects/1b2ae84b95ae44d18119450b94735f03 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:54:45.283647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.283935 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.284211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.302304 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:54:45.303519 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:45.363297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1399/2790] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:54:45.371225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.371493 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.371767 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.390561 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:54:45.391749 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:45.399638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1392/2791] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:54:45.403759 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.413356 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b66946-2fc1-44c8-b29b-75641041dc45 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.418061 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.418730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.419007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.419278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.419996 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:54:45.421029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.422447 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.433794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1400/2792] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:54:45.438571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.447934 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5baab8-a02b-4596-99e8-ddb123e56226 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.453034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.453670 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.453947 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.454253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.454941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.456820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.591279 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1393/2793] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:54:45 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:54:45.596227 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.606282 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ab47-2b48-4436-9a95-a9c040a61d85 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.661515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.662166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.662469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.662741 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.663436 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.665375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.672896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:54:45.780590 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1401/2794] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:54:45 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:54:45.784718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.796055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021c330-6d45-45aa-8684-2ef6047a7e85 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.800509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.801162 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.801441 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.801714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.802461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.804102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.814077 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1394/2795] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:54:45.818363 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.828353 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd490f9-a06c-4303-a2ec-acd0b019180e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.833013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.833659 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.833943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.834248 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.834965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.836598 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413f553-8b99-4b9a-a450-417d690913d9 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.849126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1402/2796] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:54:45.853755 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:45.864312 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696b44-96f0-4b9a-8ce0-1719de7ceef3 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:45.868972 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:45.869838 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.870124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.870593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] PATH_INFO: `/v3/projects/7c24fb658979424f9a41414ff1a9e721/users/7716368772fa40959a5d62279bee2d21/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.885346 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c24fb658979424f9a41414ff1a9e721, user_id=7716368772fa40959a5d62279bee2d21, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:45.888024 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:45.957088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1395/2797] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:54:45 2022] PUT /identity/v3/projects/7c24fb658979424f9a41414ff1a9e721/users/7716368772fa40959a5d62279bee2d21/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:54:45.968480 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:45.968784 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:45.969034 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:45.995327 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] MFA Rules not processed for user `7716368772fa40959a5d62279bee2d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:54:45.996545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:46.058959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1403/2798] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:54:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:54:46.063905 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.125092 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ad9d8-d524-46f8-8680-2147704edb2e admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.129711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.130528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.130688 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.130920 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.131595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.133486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.227738 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1396/2799] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:54:46 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:54:46.231944 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.243105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636c0dd-7dea-4a1d-abe2-1de0330c15ad None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.303823 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.304481 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.304762 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.305036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.305720 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.307679 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.391793 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1404/2800] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:54:46 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:54:46.395943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.406413 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9a71-b90d-4ef2-bb8f-1c22c2481ef4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.411015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.411666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.411948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.412223 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.413032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.414639 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.416716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:54:46.426280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1397/2801] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:54:46.430541 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.441822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438aa632-f351-48b7-9713-4335df5e2ac2 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.446769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.447655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.447937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.448220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] PATH_INFO: `/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792/users/f4d931e1711d40989ab8caf8f08add4a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.465382 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744d0d733bce4d2ba3f5c6ef08d9b792, user_id=f4d931e1711d40989ab8caf8f08add4a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.468096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.468796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.469918 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:46.540401 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1178746-71eb-437f-9ad0-10f850dfdf96 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.545860 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.546547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.546825 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.547097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.548856 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:46.561884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.563592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.566955 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1405/2802] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:54:46 2022] PUT /identity/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792/users/f4d931e1711d40989ab8caf8f08add4a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:54:46.568208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.578037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1398/2803] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:54:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:54:46.611257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:54:46.632433 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d54b4-ba25-47de-9bef-af584c523bb1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.637126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.638015 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.638341 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.638615 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.639947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.641593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799b94f-4dcf-42f9-a265-827ad8c476d7 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.651515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1406/2804] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:54:46.656267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.666599 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0ebda-3f5b-42f2-8864-cb06ff69c1aa service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.672373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.673138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.673420 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.673694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] PATH_INFO: `/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792/users/f4d931e1711d40989ab8caf8f08add4a/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.675798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744d0d733bce4d2ba3f5c6ef08d9b792, user_id=f4d931e1711d40989ab8caf8f08add4a, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.678444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.688830 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1399/2805] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:54:46 2022] PUT /identity/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792/users/f4d931e1711d40989ab8caf8f08add4a/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:54:46.698835 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.699132 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.699411 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.727245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] MFA Rules not processed for user `f4d931e1711d40989ab8caf8f08add4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:54:46.728464 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:46.800641 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1407/2806] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:54:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:54:46.811444 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:46.815588 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:46.869611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41615b9-546b-47c0-b4c3-d90eb62d4ac8 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:46.873742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:46.874416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:46.874697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:46.874970 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:46.930261 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:46.931701 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:46.941090 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1400/2807] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:54:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:54:55.402975 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:54:55.404397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:54:55.416191 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba9ee-17c4-42cc-9211-3c42cdf8e287 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:54:55.420512 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:54:55.421111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:54:55.421399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:54:55.421656 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:54:55.430352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:54:55.431756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a398cd-a1ec-45e9-96b5-cf531782df07 service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:54:55.441540 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1408/2808] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:54:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:54:56.937119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:05.437406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=54 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:27.218400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:27.279773 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c32ec2-711d-4b03-989a-e9841c6edb61 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:27.284363 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:27.285229 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:27.285509 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:27.285781 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] PATH_INFO: `/v3/users/7716368772fa40959a5d62279bee2d21` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:27.286981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7716368772fa40959a5d62279bee2d21)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:27.288900 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:27.518686 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7716368772fa40959a5d62279bee2d21'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:27.525563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7716368772fa40959a5d62279bee2d21'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:27.537635 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7716368772fa40959a5d62279bee2d21'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:27.538382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7716368772fa40959a5d62279bee2d21'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:27.631097 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7716368772fa40959a5d62279bee2d21'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:27.637620 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1401/2809] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:55:27 2022] DELETE /identity/v3/users/7716368772fa40959a5d62279bee2d21 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:27.642184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:27.642666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:27.642779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:27.661024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:27.662219 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:27.727215 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1409/2810] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:27.737629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:27.739111 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:27.798439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffdcc5-8235-495c-be5b-d8871a945dcc admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:27.801935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:27.802610 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:27.802902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:27.803181 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:27.812895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:27.814380 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:27.823902 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1402/2811] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:28.295408 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:28.296893 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:28.311479 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81657d-4f32-4122-a8c1-afe6de21397e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:28.314949 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:28.315813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:28.316230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:28.316511 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] PATH_INFO: `/v3/projects/7c24fb658979424f9a41414ff1a9e721` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:28.317698 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c24fb658979424f9a41414ff1a9e721)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:28.319730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:28.485466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c24fb658979424f9a41414ff1a9e721', 'request_id': 'req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.491744 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c24fb658979424f9a41414ff1a9e721', 'request_id': 'req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.497717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c24fb658979424f9a41414ff1a9e721', 'request_id': 'req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.618969 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] The token cache is being invalidate because project 7c24fb658979424f9a41414ff1a9e721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:55:28.619171 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.619771 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.621323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1410/2812] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:55:28 2022] DELETE /identity/v3/projects/7c24fb658979424f9a41414ff1a9e721 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 09:55:28.625977 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:28.635505 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556eaed-0a3d-43ff-a9a5-07b9081d72ad service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:28.652998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:28.691074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:28.691935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:28.692213 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:28.692484 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] PATH_INFO: `/v3/users/f4d931e1711d40989ab8caf8f08add4a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:28.693650 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d931e1711d40989ab8caf8f08add4a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:28.695711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:28.948199 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d931e1711d40989ab8caf8f08add4a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.954797 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d931e1711d40989ab8caf8f08add4a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.967126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d931e1711d40989ab8caf8f08add4a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:28.967801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d931e1711d40989ab8caf8f08add4a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.047740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-4dd397d3-53ed-4c4b-aaed-5d092f04dc3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d931e1711d40989ab8caf8f08add4a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.054048 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1403/2813] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:55:28 2022] DELETE /identity/v3/users/f4d931e1711d40989ab8caf8f08add4a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:29.512375 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:29.514431 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:29.529601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5c9c8-abd1-449c-8fb3-8f93adcbdf0f admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:29.592158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:29.593002 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:29.593278 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:29.593546 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] PATH_INFO: `/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:29.594746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744d0d733bce4d2ba3f5c6ef08d9b792)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:29.596649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:29.694297 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:29.714000 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744d0d733bce4d2ba3f5c6ef08d9b792', 'request_id': 'req-6445fb03-db03-4819-a185-753069e052f4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.720498 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744d0d733bce4d2ba3f5c6ef08d9b792', 'request_id': 'req-6445fb03-db03-4819-a185-753069e052f4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.725956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744d0d733bce4d2ba3f5c6ef08d9b792', 'request_id': 'req-6445fb03-db03-4819-a185-753069e052f4'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.801880 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] The token cache is being invalidate because project 744d0d733bce4d2ba3f5c6ef08d9b792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:55:29.802102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.802832 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6445fb03-db03-4819-a185-753069e052f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:29.804441 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1411/2814] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:55:29 2022] DELETE /identity/v3/projects/744d0d733bce4d2ba3f5c6ef08d9b792 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:29.809414 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:29.810454 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:29.810562 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:29.829078 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:29.830308 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:29.888740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1404/2815] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:29.897790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:29.898107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:29.898417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:29.917126 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:29.918352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:29.926492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1412/2816] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:29.930835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:29.940997 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4290de8-28e7-4040-95b3-e1a37fd171c7 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:29.945438 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:29.946076 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:29.946386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:29.946658 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:29.947399 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:55:29.947700 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:29.949502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:29.961433 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1405/2817] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:55:29.965724 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:29.975866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8168f4-b8cb-4627-9e10-0c06de4e9452 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:29.980578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:29.981211 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:29.981488 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:29.981757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:29.982484 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:29.984352 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.107223 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1413/2818] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:55:29 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:55:30.111278 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.121191 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00c3e-1304-4429-bc7b-3b47bb3ec96c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.146708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:30.178762 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.179431 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.179729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.179991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.180689 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.182666 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.334713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1406/2819] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:55:30 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:55:30.338899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.349746 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3f223-92ad-45a8-9abd-9115db5da518 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.355778 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.356440 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.357303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.357513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.358261 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.359865 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.370822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1414/2820] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:30.375449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.385702 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2baf5-f0c1-4630-9e2f-ee446ec73751 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.390382 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.391036 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.391315 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.391586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.392302 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.393897 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f264c-aca9-43b5-94e8-cb52e4b70c92 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.404169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1407/2821] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:30.408362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.420329 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2b15d-6944-4f66-89a7-8c0adc9c496a admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.424901 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.425775 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.426054 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.426362 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] PATH_INFO: `/v3/projects/c15279be72fb490fb8c2406bba7b2cef/users/dc2c8eae1bae4a309bc9d1b3920075f8/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.441906 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15279be72fb490fb8c2406bba7b2cef, user_id=dc2c8eae1bae4a309bc9d1b3920075f8, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.444710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.557614 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1415/2822] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:30 2022] PUT /identity/v3/projects/c15279be72fb490fb8c2406bba7b2cef/users/dc2c8eae1bae4a309bc9d1b3920075f8/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:30.568623 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.568917 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.569176 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.601063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] MFA Rules not processed for user `dc2c8eae1bae4a309bc9d1b3920075f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:30.602409 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:30.672122 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1408/2823] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:30.676597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.741819 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454751-c95f-4c92-99e8-a57f39e62aa1 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.746704 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.747390 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.747675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.747954 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.748664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.750625 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:30.832730 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1416/2824] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:55:30 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:55:30.836916 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:30.848534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66d1c-8889-4249-9a59-5d4599ef2c21 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:30.902348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:30.903012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:30.903288 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:30.904200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:30.904884 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:30.906840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.023052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1409/2825] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:55:30 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:55:31.027066 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:31.037940 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ad07e-67d9-4021-918e-0d7b8fca2864 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:31.042765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:31.043406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.043688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.043963 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.044677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:31.046285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.056722 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1417/2826] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 09:55:31.060771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:31.070638 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b866d8-5050-42c7-8902-18af7b2f334f admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:31.075296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:31.076675 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.076952 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.077221 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] PATH_INFO: `/v3/projects/c9a5002d2377456f8d2067ecac5b172c/users/3a9d254afe1040e8b9215b77060ae2a5/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.093268 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a5002d2377456f8d2067ecac5b172c, user_id=3a9d254afe1040e8b9215b77060ae2a5, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:31.095953 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.225458 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1410/2827] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:31 2022] PUT /identity/v3/projects/c9a5002d2377456f8d2067ecac5b172c/users/3a9d254afe1040e8b9215b77060ae2a5/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:31.229434 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:31.240103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0332f-c569-4b3e-938a-d1c832eb45af admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:31.260048 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:31.299468 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:31.300041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.300326 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.300603 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.301323 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:31.302961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754e179-e871-456d-96dc-e64f283d6a39 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.313095 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1418/2828] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:31.317400 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:31.322840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:31.328104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a298d2a-b8c0-4772-8864-d9404aafd8c9 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:31.333101 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:31.333967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.334289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.334555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] PATH_INFO: `/v3/projects/c9a5002d2377456f8d2067ecac5b172c/users/3a9d254afe1040e8b9215b77060ae2a5/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.339132 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a5002d2377456f8d2067ecac5b172c, user_id=3a9d254afe1040e8b9215b77060ae2a5, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:31.341784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.352726 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1411/2829] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:31 2022] PUT /identity/v3/projects/c9a5002d2377456f8d2067ecac5b172c/users/3a9d254afe1040e8b9215b77060ae2a5/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:31.363655 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.363943 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.364222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.389994 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] MFA Rules not processed for user `3a9d254afe1040e8b9215b77060ae2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:31.391222 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:31.458426 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1419/2830] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:31.469800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:31.471495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:31.528554 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a3356-e941-445d-a9cd-e747be573d15 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:31.532880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:31.533525 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:31.533803 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:31.534075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:31.543556 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:31.545000 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec3013-c8d7-4913-ba70-b1f88ba117cb service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:31.554553 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1412/2831] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:32.866076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:32.927937 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cca96-c0a2-46c4-ae2b-75a02fbb2e9e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:32.932412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:32.933264 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:32.933544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:32.933818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] PATH_INFO: `/v3/users/dc2c8eae1bae4a309bc9d1b3920075f8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:32.935022 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2c8eae1bae4a309bc9d1b3920075f8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:32.936930 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:33.164779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2c8eae1bae4a309bc9d1b3920075f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:33.165385 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2c8eae1bae4a309bc9d1b3920075f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:33.166882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:33.177947 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2c8eae1bae4a309bc9d1b3920075f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:33.178604 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2c8eae1bae4a309bc9d1b3920075f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:33.185600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2c8eae1bae4a309bc9d1b3920075f8'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:33.322312 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1420/2832] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:55:32 2022] DELETE /identity/v3/users/dc2c8eae1bae4a309bc9d1b3920075f8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:33.327182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:33.327678 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:33.328311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:33.347044 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:33.348249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:33.405817 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1413/2833] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:33.415597 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:33.417205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:33.475196 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea523a-fef9-4936-8e6b-a48678ea54ca admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:33.478950 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:33.479588 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:33.479868 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:33.480138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:33.489539 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:33.490984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:33.500917 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1421/2834] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:34.038643 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:34.040071 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:34.052448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00eb08-44c5-4bd1-b75d-916c1152f4a2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:34.056987 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:34.057844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:34.058119 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:34.058435 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] PATH_INFO: `/v3/projects/c15279be72fb490fb8c2406bba7b2cef` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:34.059613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15279be72fb490fb8c2406bba7b2cef)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:34.061528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:34.173798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15279be72fb490fb8c2406bba7b2cef', 'request_id': 'req-50f9d522-a573-46e6-98a8-b10bebb432cb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.179935 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15279be72fb490fb8c2406bba7b2cef', 'request_id': 'req-50f9d522-a573-46e6-98a8-b10bebb432cb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.186073 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15279be72fb490fb8c2406bba7b2cef', 'request_id': 'req-50f9d522-a573-46e6-98a8-b10bebb432cb'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.317225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] The token cache is being invalidate because project c15279be72fb490fb8c2406bba7b2cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:55:34.317449 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.318015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.319586 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1414/2835] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:55:34 2022] DELETE /identity/v3/projects/c15279be72fb490fb8c2406bba7b2cef => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:34.323787 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:34.334332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08731d36-823f-47b0-90d5-649e9cce14fb service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:34.392068 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:34.392919 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:34.393194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:34.393496 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] PATH_INFO: `/v3/users/3a9d254afe1040e8b9215b77060ae2a5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:34.394688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9d254afe1040e8b9215b77060ae2a5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:34.400726 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:34.639055 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9d254afe1040e8b9215b77060ae2a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.639703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9d254afe1040e8b9215b77060ae2a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.651595 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9d254afe1040e8b9215b77060ae2a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.652238 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9d254afe1040e8b9215b77060ae2a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.656611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:34.659715 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-5665d080-b236-4676-8a3d-dbbe53176154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9d254afe1040e8b9215b77060ae2a5'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:34.756350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1422/2836] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:55:34 2022] DELETE /identity/v3/users/3a9d254afe1040e8b9215b77060ae2a5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:35.209205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:35.210776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:35.225964 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9d522-a573-46e6-98a8-b10bebb432cb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:35.239660 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:35.286304 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:35.287161 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.287466 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.287717 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] PATH_INFO: `/v3/projects/c9a5002d2377456f8d2067ecac5b172c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.288886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a5002d2377456f8d2067ecac5b172c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:35.290812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:35.405789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a5002d2377456f8d2067ecac5b172c', 'request_id': 'req-36da0cea-3499-4bd4-99b3-06464098525a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:35.412521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a5002d2377456f8d2067ecac5b172c', 'request_id': 'req-36da0cea-3499-4bd4-99b3-06464098525a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:35.418573 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a5002d2377456f8d2067ecac5b172c', 'request_id': 'req-36da0cea-3499-4bd4-99b3-06464098525a'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:35.522697 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] The token cache is being invalidate because project c9a5002d2377456f8d2067ecac5b172c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:55:35.522962 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:35.523582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-36da0cea-3499-4bd4-99b3-06464098525a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:35.525142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1415/2837] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:55:35 2022] DELETE /identity/v3/projects/c9a5002d2377456f8d2067ecac5b172c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:35.530111 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.530635 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.530827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.550662 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:35.560102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:35.614178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1423/2838] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:35.623098 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.623386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.623664 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.643859 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:35.645099 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:35.653506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1416/2839] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:35.658120 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:35.668169 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353198-036f-4a26-8552-9cb6600402ff None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:35.672765 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:35.673409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.673688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.673956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.674711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:55:35.680376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:35.680376 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:35.688405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1424/2840] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 09:55:35.692771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:35.703921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4790fa-9b87-40d4-a432-b6c9718e5f92 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:35.708695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:35.709347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.709625 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.709895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.710626 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:35.712512 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:35.826646 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1417/2841] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:55:35 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:55:35.831242 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:35.841076 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101d9bc-27be-4d21-a147-746e4237b505 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:35.896217 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:35.896873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:35.897153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:35.897427 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:35.898121 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:35.900065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:36.036611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1425/2842] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:55:35 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:55:36.040683 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:36.051593 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fefa5-9bc8-4078-a153-a3753f812f19 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:36.056096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:36.056743 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:36.057037 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:36.057296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:36.058013 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:36.059645 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:36.071284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1418/2843] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:36.075273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:36.085194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cdc83-00b6-4933-966c-84cb320a347e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:36.089675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:36.090355 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:36.090636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:36.090908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:36.091617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:36.093193 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94f40a-7d64-40c3-945b-d97b4a36574c admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:36.102859 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1426/2844] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:36.106882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:36.117224 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf0bc3-d9ae-40b7-a091-fba68c0fb22c admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:36.121812 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:36.122719 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:36.123003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:36.123275 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] PATH_INFO: `/v3/projects/0a8d51ccb2dc4f66a17a51128a5dccf7/users/fd5ea896e6ba4ba098d3184d697e5f9b/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:36.138267 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8d51ccb2dc4f66a17a51128a5dccf7, user_id=fd5ea896e6ba4ba098d3184d697e5f9b, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:36.140963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:36.235095 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1419/2845] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:36 2022] PUT /identity/v3/projects/0a8d51ccb2dc4f66a17a51128a5dccf7/users/fd5ea896e6ba4ba098d3184d697e5f9b/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:36.245397 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:36.245681 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:36.245964 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:36.273651 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] MFA Rules not processed for user `fd5ea896e6ba4ba098d3184d697e5f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:36.274967 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:36.316673 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:36.338979 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1427/2846] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:36.350054 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:36.351865 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:36.397348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:36.410130 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cb2a3-6643-401b-8989-245c956b410d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:36.413989 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:36.417254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:36.417254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:36.417254 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:36.424713 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:36.426134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54b1ef-932e-4c9a-96dd-8222ad75a3ba service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:36.434993 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1420/2847] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:40.477866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:40.539101 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fd760-553b-4ded-8feb-de7a4a58f0e8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:40.543785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:40.544647 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:40.544924 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:40.545194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] PATH_INFO: `/v3/users/fd5ea896e6ba4ba098d3184d697e5f9b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:40.546413 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5ea896e6ba4ba098d3184d697e5f9b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:40.548830 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:40.784857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5ea896e6ba4ba098d3184d697e5f9b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:40.785586 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5ea896e6ba4ba098d3184d697e5f9b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:40.798389 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5ea896e6ba4ba098d3184d697e5f9b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:40.799052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5ea896e6ba4ba098d3184d697e5f9b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:40.805867 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5ea896e6ba4ba098d3184d697e5f9b'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:40.909332 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1428/2848] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 09:55:40 2022] DELETE /identity/v3/users/fd5ea896e6ba4ba098d3184d697e5f9b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:40.914424 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:40.914888 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:40.915047 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:40.932960 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:40.934135 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:40.990966 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1421/2849] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:41.000435 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:41.001861 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:41.058695 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d329860-4f34-4204-bb5f-dd47de798f3b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:41.062331 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:41.062957 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.063235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.063506 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.072324 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:41.073732 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b038e-23c5-4760-afc3-9b3e8e6e9c84 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:41.083012 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1429/2850] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:41.504840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:41.506844 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:41.519975 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a8270-91a9-40a7-8e18-56d1507ca8b4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:41.524457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:41.525345 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.525629 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.525904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] PATH_INFO: `/v3/projects/0a8d51ccb2dc4f66a17a51128a5dccf7` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.527114 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8d51ccb2dc4f66a17a51128a5dccf7)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:41.529034 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:41.666014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8d51ccb2dc4f66a17a51128a5dccf7', 'request_id': 'req-9cbfb01e-7536-4f25-99f7-669c227e339d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:41.672439 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8d51ccb2dc4f66a17a51128a5dccf7', 'request_id': 'req-9cbfb01e-7536-4f25-99f7-669c227e339d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:41.678118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8d51ccb2dc4f66a17a51128a5dccf7', 'request_id': 'req-9cbfb01e-7536-4f25-99f7-669c227e339d'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:41.749867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] The token cache is being invalidate because project 0a8d51ccb2dc4f66a17a51128a5dccf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 09:55:41.750166 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:41.750824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9cbfb01e-7536-4f25-99f7-669c227e339d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 09:55:41.752967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1422/2851] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 09:55:41 2022] DELETE /identity/v3/projects/0a8d51ccb2dc4f66a17a51128a5dccf7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:41.759518 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.760007 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.760129 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.768020 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:41.778639 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:41.779815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:41.838670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1430/2852] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:41.846547 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.846835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.847113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.865138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:41.866353 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:41.874096 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1423/2853] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:55:41.878752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:41.888113 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b460-e6fe-4c84-9420-7fc33cbfd3e0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:41.893153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:41.893800 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.894077 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.894384 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.895100 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 09:55:41.895405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:41.897010 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:41.908370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1431/2854] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 09:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 09:55:41.912596 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:41.921939 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427f652-1f7d-44ca-870f-877fea2c44e4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:41.926855 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:41.927540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:41.927820 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:41.928093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:41.928789 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:41.930722 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.014901 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1424/2855] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:55:41 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:55:42.019014 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:42.028758 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d8517-3242-4fec-a743-367db26f2956 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:42.083997 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:42.084654 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.084933 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.085205 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.085900 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:42.087862 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.168774 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1432/2856] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:55:42 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:55:42.173710 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:42.183998 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9aa22-5418-4c9f-b601-4d8aed45b1ff admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:42.188598 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:42.189836 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.190066 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.190377 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.197102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:42.197102 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.203504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1425/2857] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:42.207489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:42.218266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd8ff7-9adc-4aa5-a45c-b3de0759f47d admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:42.223165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:42.223892 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.224170 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.230057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.230057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:42.230057 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a10ba-4ff6-437e-b209-657951cef1ca admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.240017 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1433/2858] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:42.244182 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:42.255205 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb1055-e70b-4319-84d9-2b17b726e644 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:42.260138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:42.264720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.264720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.264720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] PATH_INFO: `/v3/projects/d927de989e8244d19dc9d2f54da6738c/users/43510113da604d528b4472ee77bc1061/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.277544 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d927de989e8244d19dc9d2f54da6738c, user_id=43510113da604d528b4472ee77bc1061, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:42.280178 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.352063 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1426/2859] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:42 2022] PUT /identity/v3/projects/d927de989e8244d19dc9d2f54da6738c/users/43510113da604d528b4472ee77bc1061/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:42.357406 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfe719-cbe9-4d95-8fe9-237f2cdcbc42 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.357920 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfe719-cbe9-4d95-8fe9-237f2cdcbc42 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.358420 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfe719-cbe9-4d95-8fe9-237f2cdcbc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.376401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-09cfe719-cbe9-4d95-8fe9-237f2cdcbc42 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:42.377699 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-09cfe719-cbe9-4d95-8fe9-237f2cdcbc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:42.439041 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1434/2860] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:42.448555 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:42.450443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:42.458417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:42.509471 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054cb8a-28a3-4e04-a719-cc0729cb68cf admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:42.513201 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:42.513849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:42.514126 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:42.514432 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:42.524143 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:42.525587 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:42.535711 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1427/2861] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:48.003123 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.003646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.003769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.024486 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:48.033613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] MFA Rules not processed for user `43510113da604d528b4472ee77bc1061`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:48.050533 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:48.105903 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1435/2862] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 09:55:48.111003 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.121282 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f945c5-7185-4efc-a722-c9e01e827d54 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.126504 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.127180 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.127464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.127742 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.128563 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.130398 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.251943 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1428/2863] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 09:55:48 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 09:55:48.254844 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.265521 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a158f0-fc91-47a6-a2a9-636a01fdc639 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.323887 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.324489 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.324770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.325044 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.325743 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.327799 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.454591 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1436/2864] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 09:55:48 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 09:55:48.458696 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.469464 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f121509-d15f-4943-9160-b47034e967fe admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.474704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.475365 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.475647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.475925 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.476648 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.478321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.490488 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1429/2865] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:48.494733 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.505235 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f591a2-9a0c-4fa8-bb2e-9ecbbd3fd893 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.509809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.510728 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.511006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.511276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] PATH_INFO: `/v3/projects/10ac436ee5c3471894319640d24d2833/users/1baff8a0a3044aee963796362ac56dda/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.526346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ac436ee5c3471894319640d24d2833, user_id=1baff8a0a3044aee963796362ac56dda, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.529036 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.655692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1437/2866] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:48 2022] PUT /identity/v3/projects/10ac436ee5c3471894319640d24d2833/users/1baff8a0a3044aee963796362ac56dda/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:48.660248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.670963 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244db7a-9e30-4356-ada4-095847c16441 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.730369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.732497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.733015 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.733197 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.733930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.735631 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507502a-5d30-4f68-a325-7bd064c0362d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.749882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1430/2867] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 09:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 09:55:48.751069 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:48.761881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9dabd-afad-436c-94b4-2e0478d39659 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:48.767882 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:48.768834 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:48.769184 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:48.769548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] PATH_INFO: `/v3/projects/10ac436ee5c3471894319640d24d2833/users/1baff8a0a3044aee963796362ac56dda/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:48.771634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ac436ee5c3471894319640d24d2833, user_id=1baff8a0a3044aee963796362ac56dda, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:48.774505 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:48.786270 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1438/2868] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 09:55:48 2022] PUT /identity/v3/projects/10ac436ee5c3471894319640d24d2833/users/1baff8a0a3044aee963796362ac56dda/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 09:55:54.546776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d460-6821-44f8-8980-bcdb32d05095 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:54.547376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d460-6821-44f8-8980-bcdb32d05095 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:54.547376 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d460-6821-44f8-8980-bcdb32d05095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:54.578827 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-aa01d460-6821-44f8-8980-bcdb32d05095 None None] MFA Rules not processed for user `1baff8a0a3044aee963796362ac56dda`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:54.580777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-aa01d460-6821-44f8-8980-bcdb32d05095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:54.645233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:54.654608 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1431/2869] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:55:54.671165 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:55:54.673097 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:54.744593 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:55:54.771234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67d4f-8204-4183-8902-8b5fb265510b admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:55:54.777330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:55:54.782334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:54.782334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:54.782334 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:54.842609 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:55:54.844272 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:55:54.853792 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1439/2870] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:55:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:55:57.664352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:55:57.664948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:55:57.664948 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:55:57.683222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] MFA Rules not processed for user `43510113da604d528b4472ee77bc1061`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 09:55:57.686523 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:55:57.691347 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1432/2871] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 09:56:05.087756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=32 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:56:07.678041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=5 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:56:19.844358 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:56:19.845757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:19.901256 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6cc5-dc9b-466e-9913-d20dc89301e0 service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:56:19.905585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:56:19.906225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:56:19.906545 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:56:19.906818 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:56:19.908630 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:19.918618 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:56:19.920090 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:56:19.928058 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1440/2872] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:56:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:56:21.738084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:56:21.739477 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:21.807669 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b5de5-b155-4537-acd7-7fe9c503ac72 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:56:21.813188 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:56:21.813834 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:56:21.814118 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:56:21.814423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:56:21.816200 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:21.828793 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:56:21.830332 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:56:21.840835 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1433/2873] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:56:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:56:29.924460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=26 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:56:31.836369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=32 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:56:39.519281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:56:39.532313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73a382-4444-4803-bfe6-8eae433a901c service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:56:39.537890 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:56:39.538556 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:56:39.538837 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:56:39.539112 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:56:39.541178 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:39.553888 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:56:39.555456 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:56:39.566671 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1441/2874] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:56:42.438526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:56:42.448301 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d1cf-6540-4518-8773-077e92348703 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:56:42.451582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:56:42.452214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:56:42.452497 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:56:42.452772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:56:42.454611 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:56:42.468519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:56:42.469950 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:56:42.481457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1434/2875] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:56:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:56:42.768279 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:56:42.777827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae1b3-681f-49fe-9480-456a94a3c83b service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:56:42.781402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:56:42.782029 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:56:42.782345 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:56:42.782664 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:56:42.792529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:56:42.793983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:56:42.803253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1442/2876] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:56:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:56:52.476472 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:56:52.799168 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=12 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:57:41.223866 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:57:41.233348 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd330710-1e43-46b4-8404-c6261ff1922f service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:57:41.237089 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:57:41.237715 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:57:41.237991 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:57:41.238292 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:57:41.240017 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:57:41.249578 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:57:41.251019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:57:41.259009 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1435/2877] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:57:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:57:51.255368 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:59:00.460731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:59:00.497711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bb51-df82-4923-b800-fe2685a2a839 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:59:00.501337 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:59:00.501966 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:59:00.502273 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:59:00.502553 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:59:00.504322 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:59:00.548105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:59:00.549529 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:59:00.560973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1443/2878] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:59:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:59:10.555515 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=31 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:59:18.464804 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:59:18.500265 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ab577-ef52-4f42-97f9-b1d6e89ed19c service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:59:18.503904 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:59:18.504534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:59:18.504815 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:59:18.505088 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:59:18.506880 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:59:18.529025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:59:18.530419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:59:18.542271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1436/2879] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:59:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:59:18.910866 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:59:18.921606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f64a8-0b6f-4026-b02d-752a935b4016 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:59:18.925896 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:59:18.926815 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:59:18.929814 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:59:18.930110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:59:18.932719 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:59:18.943064 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:59:18.944491 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:59:18.953779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1444/2880] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:59:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 09:59:19.036653 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:59:19.046156 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c1cc-4d85-4a93-91ec-4b9787870b16 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:59:19.049824 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:59:19.050483 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:59:19.050770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:59:19.051032 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:59:19.052784 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:59:19.066794 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:59:19.068172 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:59:19.079072 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1437/2881] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:59:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 09:59:28.950444 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:59:29.074253 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=29 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 09:59:50.509559 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 09:59:50.531632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311481fc-71b8-493f-b99a-d3a6b560ae96 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 09:59:50.536581 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 09:59:50.537221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 09:59:50.537514 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 09:59:50.537794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 09:59:50.539899 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 09:59:50.565571 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 09:59:50.567202 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 09:59:50.578106 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1445/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 09:59:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:00:00.572968 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=15 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:04.057514 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:04.066428 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02cece-dbd2-45c4-8d64-406520651a08 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:04.069982 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:04.070641 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:04.070921 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:04.071192 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:04.080407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:04.081839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:04.090906 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1438/2883] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:04.997945 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:05.016828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:05.016828 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4576ac-abdf-4d66-9836-1b4fa37608c5 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:05.023640 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:05.024361 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:05.024646 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:05.024923 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:05.026768 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:05.041613 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:05.047565 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:05.055474 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1446/2884] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:07.619134 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:07.631994 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb658291-d6a0-4d6d-b5ff-ab26cad0a4d7 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:07.637393 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:07.638020 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:07.638334 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:07.638705 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:07.640455 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:07.652613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:07.654190 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:07.665621 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1439/2885] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:10.372677 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:10.382779 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87571-7bb8-4f72-8772-7d78b43d8b79 service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:10.386401 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:10.387056 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:10.387338 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:10.387611 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:10.455785 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:10.457208 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:10.475916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1447/2886] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:13.975956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:13.975956 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:13.992614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970d66d-e51c-4da9-afb0-e7015af1975a service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:13.997882 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:13.998792 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:13.999137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:13.999534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:14.001976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:14.014582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:14.016186 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:14.026447 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1440/2887] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:17.521752 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:17.533973 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddee8f-2e7a-42c4-8f57-afc35951e5d4 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:17.539578 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:17.545494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:17.545494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:17.545494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:17.545494 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:17.555285 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:17.557675 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:17.568991 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1448/2888] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:17.636461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:17.647357 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf21bff-415a-4784-8546-82e27aa09bcc service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:17.652427 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:17.653286 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:17.653591 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:17.653908 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] PATH_INFO: `/v3/users/43510113da604d528b4472ee77bc1061` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:17.655104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43510113da604d528b4472ee77bc1061)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:17.657289 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:17.835693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43510113da604d528b4472ee77bc1061'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:17.842457 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43510113da604d528b4472ee77bc1061'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:17.854222 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43510113da604d528b4472ee77bc1061'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:17.854885 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43510113da604d528b4472ee77bc1061'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:17.925698 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43510113da604d528b4472ee77bc1061'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:17.931932 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1441/2889] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 10:01:17 2022] DELETE /identity/v3/users/43510113da604d528b4472ee77bc1061 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:18.147914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:18.163142 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f788c17-530b-48eb-bb59-eabd7b251b77 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:18.220648 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:18.221492 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:18.221769 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:18.222042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] PATH_INFO: `/v3/projects/d927de989e8244d19dc9d2f54da6738c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:18.223250 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d927de989e8244d19dc9d2f54da6738c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:18.225141 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:18.300003 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd927de989e8244d19dc9d2f54da6738c', 'request_id': 'req-05db59a9-ba87-495c-8cde-81c622eba1ac'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.301871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:18.306207 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd927de989e8244d19dc9d2f54da6738c', 'request_id': 'req-05db59a9-ba87-495c-8cde-81c622eba1ac'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.311727 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd927de989e8244d19dc9d2f54da6738c', 'request_id': 'req-05db59a9-ba87-495c-8cde-81c622eba1ac'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.400001 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] The token cache is being invalidate because project d927de989e8244d19dc9d2f54da6738c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 10:01:18.400225 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.400850 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.402410 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1449/2890] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 10:01:18 2022] DELETE /identity/v3/projects/d927de989e8244d19dc9d2f54da6738c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:18.406373 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:18.407771 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:18.420770 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7957b-5ab4-4032-8037-afc628605a6d admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:18.475938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:18.476801 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:18.477080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:18.477354 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] PATH_INFO: `/v3/users/1baff8a0a3044aee963796362ac56dda` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:18.478550 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1baff8a0a3044aee963796362ac56dda)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:18.480461 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:18.645603 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:18.654386 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1baff8a0a3044aee963796362ac56dda'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.661720 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1baff8a0a3044aee963796362ac56dda'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.673360 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1baff8a0a3044aee963796362ac56dda'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.674028 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1baff8a0a3044aee963796362ac56dda'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.737256 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-cd1335f3-0f0b-4b32-98c6-2dcb3061c0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1baff8a0a3044aee963796362ac56dda'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:18.743281 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1442/2891] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 10:01:18 2022] DELETE /identity/v3/users/1baff8a0a3044aee963796362ac56dda => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:19.021678 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.023234 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:19.036005 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db59a9-ba87-495c-8cde-81c622eba1ac admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.092252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.093102 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.093381 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.093653 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] PATH_INFO: `/v3/projects/10ac436ee5c3471894319640d24d2833` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.094876 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ac436ee5c3471894319640d24d2833)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.096780 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.217756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ac436ee5c3471894319640d24d2833', 'request_id': 'req-27e79110-ac35-41dd-b518-861b94f159f2'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:19.223910 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ac436ee5c3471894319640d24d2833', 'request_id': 'req-27e79110-ac35-41dd-b518-861b94f159f2'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:19.229542 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ac436ee5c3471894319640d24d2833', 'request_id': 'req-27e79110-ac35-41dd-b518-861b94f159f2'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:19.320412 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] The token cache is being invalidate because project 10ac436ee5c3471894319640d24d2833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 10:01:19.320694 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:19.321303 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-27e79110-ac35-41dd-b518-861b94f159f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:01:19.322881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1450/2892] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 10:01:19 2022] DELETE /identity/v3/projects/10ac436ee5c3471894319640d24d2833 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 10:01:19.327852 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.328142 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.328418 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.384526 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:19.385718 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:19.443456 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1443/2893] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:01:19.451065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.451350 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.451626 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.470081 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:19.471300 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:19.479114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1451/2894] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:01:19.483041 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.492495 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555c4a2-fd53-4828-9c5f-59820b3b6b8f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.497007 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.497649 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.497926 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.498214 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.498946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 10:01:19.499252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.501417 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.512582 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1444/2895] 10.222.0.48 () {58 vars in 1152 bytes} [Fri May 27 10:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 10:01:19.516575 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.525455 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c29c-37b2-43d2-99c1-6cf874b11443 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.530027 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.530697 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.530977 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.531253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.532460 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.534409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.638577 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1452/2896] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 10:01:19 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 10:01:19.642249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.652307 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348798-d02b-44b4-a93f-43aa3bfce166 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.709138 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.714538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.714538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.714538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.714538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.714538 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.779476 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:19.785171 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1445/2897] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 10:01:19 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 10:01:19.789560 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.800370 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becca87-7b22-48e8-a5e5-17f241531ed0 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.804984 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.805627 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.805914 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.806186 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.806928 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.812021 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.819387 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1453/2898] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 10:01:19.823372 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.833849 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bd4d-8199-4a82-aeb7-6051dc91affb admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.838479 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.839151 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.839434 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.839708 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.840426 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.842012 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ae858-477b-4039-b2a0-e8a1008f42fc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.851777 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1446/2899] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 10:01:19.855770 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:19.866074 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3eac4-4aa4-41de-9af9-4a3efcc30ac3 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:19.870688 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:19.871544 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.871827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.872107 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] PATH_INFO: `/v3/projects/fb894beeec434e34b0a8224128eda0c1/users/ce5ba392d54f4d9588b7e3f891e8e331/roles/8ff134ce9e0f40f38af26ff674f3b83f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:19.887333 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb894beeec434e34b0a8224128eda0c1, user_id=ce5ba392d54f4d9588b7e3f891e8e331, role_id=8ff134ce9e0f40f38af26ff674f3b83f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:19.889959 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:19.892983 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:19.979024 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1454/2900] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 10:01:19 2022] PUT /identity/v3/projects/fb894beeec434e34b0a8224128eda0c1/users/ce5ba392d54f4d9588b7e3f891e8e331/roles/8ff134ce9e0f40f38af26ff674f3b83f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:19.984021 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca799356-96c6-4765-ac55-2f97d3cc0fff None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:19.984361 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca799356-96c6-4765-ac55-2f97d3cc0fff None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:19.984654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca799356-96c6-4765-ac55-2f97d3cc0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:20.003112 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ca799356-96c6-4765-ac55-2f97d3cc0fff None None] MFA Rules not processed for user `c1f81d15d7c54a30853ee3ff06b25bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:20.004311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ca799356-96c6-4765-ac55-2f97d3cc0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:20.061137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1447/2901] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 10:01:20.071110 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:20.072864 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:20.128039 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee760e41-3722-47a6-a76e-2eede5784585 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:20.131702 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:20.132336 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:20.132616 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:20.132889 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:20.141692 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:20.143138 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:20.152283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1455/2902] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:25.205369 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.205912 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.206029 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.237202 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] MFA Rules not processed for user `ce5ba392d54f4d9588b7e3f891e8e331`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:25.238613 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:25.315046 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1448/2903] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:01:25.319809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.332013 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ea6ee-d489-458b-b969-807aff707879 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.346127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.346127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.346127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.346127 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.346650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.346650 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.427822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1456/2904] 10.222.0.48 () {60 vars in 1152 bytes} [Fri May 27 10:01:25 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 10:01:25.431954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.444113 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b9de-ff20-427b-bb5f-8c29abf60689 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.504116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.504800 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.505080 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.505355 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.506049 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.508160 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.616592 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1449/2905] 10.222.0.48 () {60 vars in 1143 bytes} [Fri May 27 10:01:25 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 10:01:25.620827 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.635911 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330f158-9479-4f7e-93da-35a34dfc79ae admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.640617 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.641373 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.642466 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.642871 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.643789 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.645402 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.657448 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1457/2906] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 10:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 10:01:25.661733 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.672328 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb0d2-2038-4c47-9b11-2104f267f3f4 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.677106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.677996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.678330 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.678605 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] PATH_INFO: `/v3/projects/0db677c3a1d34d3eadb57516698bc53a/users/740a5346f4fc46848c7a665cab9b92bd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.681104 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:25.695425 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db677c3a1d34d3eadb57516698bc53a, user_id=740a5346f4fc46848c7a665cab9b92bd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.698329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.795521 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1450/2907] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 10:01:25 2022] PUT /identity/v3/projects/0db677c3a1d34d3eadb57516698bc53a/users/740a5346f4fc46848c7a665cab9b92bd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:25.799729 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.810271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e40a4-536d-4946-a57b-886926d3a44e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.872585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.873252 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.873532 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.873806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.874562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.876810 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794488-9801-4fba-a826-0069f8f84005 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.890623 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1458/2908] 10.222.0.48 () {58 vars in 1121 bytes} [Fri May 27 10:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 10:01:25.891936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:25.902706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553b956-c3ed-47f0-bc04-7edb86b41423 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:25.908108 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:25.909124 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:25.909402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:25.909671 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] PATH_INFO: `/v3/projects/0db677c3a1d34d3eadb57516698bc53a/users/740a5346f4fc46848c7a665cab9b92bd/roles/2cf0ecb1f1344e92aec134e29d0777e2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:25.911663 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db677c3a1d34d3eadb57516698bc53a, user_id=740a5346f4fc46848c7a665cab9b92bd, role_id=2cf0ecb1f1344e92aec134e29d0777e2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:25.919093 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:25.926788 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1451/2909] 10.222.0.48 () {60 vars in 1482 bytes} [Fri May 27 10:01:25 2022] PUT /identity/v3/projects/0db677c3a1d34d3eadb57516698bc53a/users/740a5346f4fc46848c7a665cab9b92bd/roles/2cf0ecb1f1344e92aec134e29d0777e2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:01:31.291257 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b413f-701a-4371-aacd-0c7ec677a1cf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:31.291807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b413f-701a-4371-aacd-0c7ec677a1cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:31.291807 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b413f-701a-4371-aacd-0c7ec677a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:31.319475 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-259b413f-701a-4371-aacd-0c7ec677a1cf None None] MFA Rules not processed for user `740a5346f4fc46848c7a665cab9b92bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:31.320689 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-259b413f-701a-4371-aacd-0c7ec677a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:31.370450 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:31.399936 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1459/2910] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:01:31.408565 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:31.410394 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:31.488754 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bf929-bd6c-4e7f-8ca7-0cb7dafbb486 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:31.494746 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:31.495402 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:31.495798 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:31.501177 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:31.555329 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:31.556969 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:31.567300 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1452/2911] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:41.493524 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:41.561976 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=68,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:01:54.919093 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:54.919596 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:54.919716 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:54.937204 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] MFA Rules not processed for user `ce5ba392d54f4d9588b7e3f891e8e331`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:01:54.938422 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:54.945183 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1460/2912] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:01:54.981545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:54.982949 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:55.060839 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b8bd-5999-42b3-8a15-b4ee7774db11 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:55.064442 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:55.065075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:55.065352 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:55.065622 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:55.067403 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:55.101084 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:55.102515 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:55.122339 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1453/2913] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:55.404806 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:55.406190 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:55.463517 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372b160-9048-4c28-b711-5b90d54e367c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:55.467098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:55.467731 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:55.468016 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:55.468294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:55.470105 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:55.479417 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:55.480885 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:55.488916 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1461/2914] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:01:56.180780 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:01:56.182271 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:56.252999 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f636bc-ddd0-456a-9036-e7dff22c2645 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:01:56.257899 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:01:56.258590 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:01:56.258867 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:01:56.259137 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:01:56.260858 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:01:56.272938 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:01:56.274540 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:01:56.284248 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1454/2915] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:02:05.485986 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=29 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:02:06.279983 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=79 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:02:11.386377 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:02:11.387849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:02:11.403667 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b1ec-c1f4-4dd2-b8c6-3635412f37c6 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:02:11.410114 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:02:11.410711 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:02:11.410989 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:02:11.411301 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:02:11.413245 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:02:11.426344 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:02:11.427980 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:02:11.438708 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1462/2916] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:02:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:02:14.938019 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:02:14.947694 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef22d41-eb62-4eec-9be9-aa3c724920a0 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:02:14.951251 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:02:14.951873 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:02:14.952149 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:02:14.952419 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:02:14.961218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:02:14.962695 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:02:14.972321 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1455/2917] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:02:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:02:21.434088 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:02:24.967735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:02:47.214881 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:02:47.224443 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90db52-b0ec-4d99-86cc-50ce1e9cb17e service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:02:47.228050 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:02:47.228710 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:02:47.228992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:02:47.229266 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:02:47.231063 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:02:47.240439 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:02:47.241849 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5d7d3-f689-4e39-a5be-799fae04aa94 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:02:47.249833 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1463/2918] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:02:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:02:57.246180 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:04:30.949423 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:04:30.958704 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7a242-738f-4dc5-8664-2246c1469637 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:04:30.962296 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:04:30.962954 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:04:30.963235 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:04:30.963907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:04:30.965218 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:04:30.979448 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:04:30.980930 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:04:30.992647 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1456/2919] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:04:35.564330 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:04:35.564972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:04:35.564972 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:04:35.583221 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] MFA Rules not processed for user `ce5ba392d54f4d9588b7e3f891e8e331`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:04:35.584394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:04:35.591608 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1464/2920] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:04:35.896170 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:04:35.904712 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba000a-7302-4ccb-8919-dcf65280e054 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:04:35.909055 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:04:35.909691 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:04:35.909971 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:04:35.910280 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:04:35.912319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:04:35.921886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:04:35.923936 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:04:35.931967 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1457/2921] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:04:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:04:45.577283 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:04:45.929022 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=12 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:04:53.477265 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:04:53.478822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:04:53.495006 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c25f8-4a7c-4951-bf81-05ce390ec722 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:04:53.500293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:04:53.501218 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:04:53.501583 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:04:53.501796 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:04:53.504501 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:04:53.517548 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:04:53.519098 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e507cf-14ce-4387-884a-adb0628938c4 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:04:53.529294 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1465/2922] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:04:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:05:03.524794 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:06:47.903227 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:06:47.904616 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:06:47.916350 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e903b58-5184-4623-8ac6-038f4a7b78fc service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:06:47.919878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:06:47.920486 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:06:47.920765 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:06:47.921127 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:06:47.929714 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:06:47.931228 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:06:47.940893 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1458/2923] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:06:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:06:57.936117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=6 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:06:59.716666 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:06:59.717188 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:06:59.717313 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:06:59.736194 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] MFA Rules not processed for user `ce5ba392d54f4d9588b7e3f891e8e331`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 10:06:59.737562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:06:59.745220 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1466/2924] 10.222.0.48 () {58 vars in 957 bytes} [Fri May 27 10:06:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 10:06:59.784053 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:06:59.793025 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e0926-b3f1-467f-ac43-ee9f5d78527e service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:06:59.796716 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3963b8bacbc74ef2a579be684a219ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:06:59.797344 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:06:59.797618 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:06:59.797886 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:06:59.809215 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:06:59.810816 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:06:59.822169 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1459/2925] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:06:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 10:07:00.115405 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:00.125293 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d98eb-25a1-4f56-8952-74ecd8299563 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:00.128956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:00.129585 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:00.129873 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:00.130153 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:00.137563 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:00.139060 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:00.147670 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1467/2926] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:07:01.083832 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:01.104571 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da099a7-6f6d-458f-829e-f8686c09c5f4 service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:01.112680 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:01.113913 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:01.114533 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:01.115116 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:01.132564 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:01.135687 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:01.149519 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1460/2927] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 10:07:05.362115 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:05.388275 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588fe3f-ac34-4c1c-98b3-ddad80e15916 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:05.394568 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:05.395230 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:05.395645 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:05.395829 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:05.397798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:07:05.412253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:05.413857 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:05.426185 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1468/2928] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:07:11.145259 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=12 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:07:15.420584 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=15 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:07:16.862408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:16.878759 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedb93b-6874-4da8-a849-be46ecbded79 service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:16.885740 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:16.888482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:16.888482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:16.888482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:16.897157 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:16.898772 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:16.909252 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1461/2929] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:07:20.133080 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:20.143756 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee372-2e25-493b-9851-d1c6abdf457e service nova] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:20.147320 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:20.147961 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:20.148246 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:20.148516 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:20.158135 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:20.159601 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:20.170276 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1469/2930] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:07:22.204758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:22.216840 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3bf66-aacd-4e7e-a87e-d346e332fe6b service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:22.228589 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55612ce26632413ebd0dfd5fc081b656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:22.229249 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:22.229528 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:22.229796 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:22.231659 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:07:22.247764 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:22.249415 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:22.268654 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1462/2931] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:07:30.165311 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:07:32.256706 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=12 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:07:51.222299 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:07:51.231291 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada3546-6f67-4a59-9e19-630713a522b3 service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:07:51.244042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:07:51.244042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:07:51.244042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:07:51.244042 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:07:51.244513 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:07:51.245158 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:07:51.253813 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1470/2932] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:07:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:08:01.249847 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=6 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:09:26.625941 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:26.664892 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d1df-a100-4983-a9bc-b310fc81abb3 service nova] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:26.668895 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07600562586a47a696ec87ef60838559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:26.669758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:26.670140 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:26.670569 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:26.753532 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:26.755016 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:26.765693 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1463/2933] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:09:30.397606 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:30.423469 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf4c9f-88df-4ad4-936c-706fd93fc48f service neutron] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:30.428610 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:30.429253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:30.429562 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:30.429809 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:30.432908 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:09:30.456790 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:30.458394 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:30.469087 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1471/2934] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:09:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:09:33.976595 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:33.988502 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c51b-3e28-4edf-a5a6-3059db9b17b4 service neutron] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:33.994284 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:33.994933 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:33.995210 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:33.995482 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:33.997233 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:09:34.010407 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:34.012498 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:34.023337 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1464/2935] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:09:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:09:34.135642 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:34.146703 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd8ed-6230-455e-a0f8-aa395e239152 service placement] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:34.152335 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:34.153286 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:34.153644 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:34.153990 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] PATH_INFO: `/v3/users/ce5ba392d54f4d9588b7e3f891e8e331` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:34.155346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5ba392d54f4d9588b7e3f891e8e331)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:34.157368 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:34.327371 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5ba392d54f4d9588b7e3f891e8e331'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:34.328103 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5ba392d54f4d9588b7e3f891e8e331'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:34.340636 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5ba392d54f4d9588b7e3f891e8e331'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:34.345065 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5ba392d54f4d9588b7e3f891e8e331'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:34.348072 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5ba392d54f4d9588b7e3f891e8e331'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:34.433965 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1472/2936] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 10:09:34 2022] DELETE /identity/v3/users/ce5ba392d54f4d9588b7e3f891e8e331 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:09:34.668729 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:34.680050 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bddf2a-091a-45ec-83b3-2f1b42ae9c1b service placement] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:34.738094 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:34.739038 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:34.739341 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:34.739614 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] PATH_INFO: `/v3/projects/fb894beeec434e34b0a8224128eda0c1` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:34.740927 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb894beeec434e34b0a8224128eda0c1)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:34.742907 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:35.003463 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:09:35.012014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb894beeec434e34b0a8224128eda0c1', 'request_id': 'req-2a247ccc-723b-4ca8-ace4-a988684b8ac3'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.018225 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb894beeec434e34b0a8224128eda0c1', 'request_id': 'req-2a247ccc-723b-4ca8-ace4-a988684b8ac3'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.024039 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb894beeec434e34b0a8224128eda0c1', 'request_id': 'req-2a247ccc-723b-4ca8-ace4-a988684b8ac3'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.095506 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] The token cache is being invalidate because project fb894beeec434e34b0a8224128eda0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 10:09:35.096117 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.096814 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.098776 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1465/2937] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 10:09:34 2022] DELETE /identity/v3/projects/fb894beeec434e34b0a8224128eda0c1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:09:35.103600 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:35.105155 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:09:35.119166 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d6b81-f8cc-421c-9c64-62373e237b66 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:35.179271 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:35.180281 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:35.181367 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:35.181634 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] PATH_INFO: `/v3/users/740a5346f4fc46848c7a665cab9b92bd` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:35.182855 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=740a5346f4fc46848c7a665cab9b92bd)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:35.184822 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:35.385253 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:09:35.390570 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740a5346f4fc46848c7a665cab9b92bd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.391231 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740a5346f4fc46848c7a665cab9b92bd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.403798 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740a5346f4fc46848c7a665cab9b92bd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.404438 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740a5346f4fc46848c7a665cab9b92bd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.411399 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740a5346f4fc46848c7a665cab9b92bd'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.484757 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1473/2938] 10.222.0.48 () {58 vars in 1223 bytes} [Fri May 27 10:09:35 2022] DELETE /identity/v3/users/740a5346f4fc46848c7a665cab9b92bd => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:09:35.749074 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:09:35.750545 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:09:35.763758 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a247ccc-723b-4ca8-ace4-a988684b8ac3 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:09:35.822981 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1f81d15d7c54a30853ee3ff06b25bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '822861f091c54d5980ee4872d75a26fa', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:09:35.823868 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:09:35.824146 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:09:35.824416 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] PATH_INFO: `/v3/projects/0db677c3a1d34d3eadb57516698bc53a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:09:35.825735 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db677c3a1d34d3eadb57516698bc53a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:09:35.827795 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:09:35.931052 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db677c3a1d34d3eadb57516698bc53a', 'request_id': 'req-389108b0-df23-4fa2-b6ad-ee39d29ac771'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.937833 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db677c3a1d34d3eadb57516698bc53a', 'request_id': 'req-389108b0-df23-4fa2-b6ad-ee39d29ac771'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:35.943996 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db677c3a1d34d3eadb57516698bc53a', 'request_id': 'req-389108b0-df23-4fa2-b6ad-ee39d29ac771'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:36.020311 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] The token cache is being invalidate because project 0db677c3a1d34d3eadb57516698bc53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80419) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 10:09:36.021204 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:36.021319 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 10:09:36.023014 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1466/2939] 10.222.0.48 () {58 vars in 1232 bytes} [Fri May 27 10:09:35 2022] DELETE /identity/v3/projects/0db677c3a1d34d3eadb57516698bc53a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 10:09:45.650649 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:09:46.016946 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:10:00.466956 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:10:00.468346 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:10:00.559476 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976094af-6629-4d3e-8f0c-145c89bf7daa admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:10:00.622925 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:10:00.623564 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:10:00.623846 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:10:00.624119 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:10:00.626052 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:10:00.638409 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:10:00.639992 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe99ee6-0934-464d-a7ab-1803786b29c5 service placement] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:10:00.650632 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1474/2940] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:10:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 10:10:10.645717 dv-cinder-825492-18-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=67 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 10:15:03.425878 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 10:15:03.427395 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 10:15:03.445534 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389108b0-df23-4fa2-b6ad-ee39d29ac771 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 10:15:03.451443 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc3407db3d994b17851618218efcb7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010e30be1e8a43ceaa3bede3ac4c9a90', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 10:15:03.452106 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 10:15:03.452391 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 10:15:03.452665 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 10:15:03.462408 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 10:15:03.464060 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fd83e-5cf2-404d-88aa-abbac2deffe5 service placement] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 10:15:03.475075 dv-cinder-825492-18-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1467/2941] 10.222.0.48 () {60 vars in 1365 bytes} [Fri May 27 10:15:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)